JPregel

Source Code

GitHub lets you download the source in multiple formats

You can also clone the project with Git by running:

$ git clone git://github.com/kowshik/JPregel

How-to build JPregel system and dev packages :

Download the source code and extract the archive.
Modify build.xml to configure hosts in your compute cluster. Setup a headless user account for SSHing in your cluster, and update the same in the build.xml.
Run the following command to compile :

$ ant compile
Run the following command to build JPregel. The JAR file : jpregel.jar is created in dist/.

$ ant build_jpregel
Run the following command to create JPregel Dev Package. The JAR file : jpregel_dev.jar is created in dist/.

$ ant build_dev

How-to create your JPregel application and deploy the system :

Extend class api.Vertex to create your own Vertex class. Lets call it ClientVertex.java.
Compile ClientVertex.java using the command :

$ javac -cp ClientVertex.java
Build application.jar using the command :

$ jar cvf application.jar

Example : jar cvf application ClientVertex.class
Copy your input graph file (adjacency list representation) to graph.dat under the same folder. Alternatively, you can also generate random graphs using the command :

$ ant runGraphGenerator -Dmax_cost=<max_cost_value> -Dnum_vertices=<num_vertices_in_graph>

Example : ant runGraphGenerator -Dmax_cost=1000 -Dnum_vertices=25, generates graph.dat file containing adjacency list of a directed graph of 25 vertices with cost of each outgoing edge from a vertex lying between 1 and 1000.
Run a remote JPregel master using the command :

$ ant runRemoteMaster -Dusername=<SSH_username> -Dapp=<absolute_path_to_application.jar> -Dvertex_class=<your_vertex_class> -Dnum_machines=<num_worker_machines>

Example : ant runRemoteMaster -Dusername=blahblah -Dapp=/home/blahblah/application.jar -Dvertex_class=ClientVertex -Dnum_machines=16
Run remote JPregel workers using the command (XX refers to number of worker managers. See "ant-projecthelp"):

$ ant runXXWkrManagers -Dusername=<SSH_username> -Dapp=<absolute_path_to_application.jar> -Dvertex_class=<your_vertex_class>

Example : ant run16WkrManagers -Dusername=blahblah -Dapp=/home/blahblah/application.jar -Dvertex_class=ClientVertex
Once the specified machines complete registration with the master, the execution of your application will commence. Logs will be written to the folder logs/. Checkpointing data is written to checkpoints/. Solutions is written to solutions/ upon completion of computations.