Installation / Deployment

Build egg file

$ cd spooq2
$ python bdist_egg

The output is stored as dist/Spooq2-<VERSION_NUMBER>-py2.7.egg

Build zip file

$ cd spooq2
$ rm
$ zip -r src/spooq2
$ mv Spooq2_$(grep "__version__" src/spooq2/ | \
    cut -d " " -f 3 | tr -d \").zip

The output is stored as Spooq2-<VERSION_NUMBER>.zip.

Include pre-build package (egg or zip) with Spark

For Submitting or Launching Spark:

$ pyspark --py-files Spooq2-<VERSION_NUMBER>.egg

The library still has to be imported in the pyspark application!

Within Running Spark Session:

>>> sc.addFile("Spooq2-<VERSION_NUMBER>.egg")
>>> import spooq2

Install local repository as package

$ cd spooq2
$ python install

Install Spooq2 directly from git

$ pip install git+

Development, Testing, and Documenting

Please refer to Setup for Development, Testing, Documenting.