Add end-to-end tutorial for EgoBipartiteGraphSAGE model. (#216)

1. Trained with GraphLearn-Training: implement model with u2i2i and
   i2i2i meta-path;
2. Infered user embedding with Dynamic-Graph-Service:
   - Add getQuery http request;
   - Refine EgoGraph/EgoTensor for parsing sampled fbs records;
   - Refine App and add uts.

Author: Seventeen17

.readthedocs.yaml

@@ -19,7 +19,7 @@ build:
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:
-   configuration: docs/en/conf.py
+   configuration: docs/conf.py
 
 # If using Sphinx, optionally build your docs in additional formats such as PDF
 # formats:

docs/en/Makefile docs/Makefile

@@ -14,16 +14,16 @@
 import sys
 
 os.system("mkdir -p apis/apis/apis/")
-os.system("javadoc -d apis/apis/apis/ -sourcepath ../../dynamic_graph_service/gsl_client/src/main/java/ -classpath ../../dynamic_graph_service/gsl_client/src/main/java/ org.aliyun.gsl_client")
-
-
+os.system("javadoc -d apis/apis/apis/ -sourcepath ../dynamic_graph_service/gsl_client/src/main/java/ -classpath ../dynamic_graph_service/gsl_client/src/main/java/ org.aliyun.gsl_client")
 
 # -- Project information -----------------------------------------------------
 
 project = 'GraphLearn'
 copyright = '2022, Alibaba-inc PAI'
 author = 'Alibaba-inc PAI'
 
+root_doc = "index_" + str(os.environ.get("READTHEDOCS_LANGUAGE"))
+
 # The full version, including alpha/beta/rc tags
 release = '2.0.0'
 

docs/en/apis/index.rst docs/apis/index.rst

@@ -176,5 +176,8 @@ after all dataloader instances have set the barrier.
 
 Users can also check barrier status on their gsl-clients:
 ```java
-    TODO(@Seventeen17): add the java example code here
+// public interface Graph
+Status checkBarrier(String name) throws UserException;
+
+// if Status.ok(), then the given barrier is READY.
 ```

docs/en/conf.py docs/conf.py

@@ -36,7 +36,7 @@ helm install my-release dgs/dgs \
 ```
 
 The graph schema must be specified from a json string or file by parameter `graphSchema`.
-A [template](https://github.com/alibaba/graph-learn/blob/master/dynamic_graph_service/conf/schema.template.json)
+An [example](https://github.com/alibaba/graph-learn/blob/master/dynamic_graph_service/conf/u2i/schema.u2i.json)
 schema file can be followed to write your customized graph schema.
 
 The info of `dl2spl` and `spl2srv`  of your pre-deployed kafka cluster should be configured when you install the chart,

docs/en/dgs/dataloader.md

@@ -6,13 +6,14 @@ This document is an e2e tutorial of offline training and online inference for a
   - Tensorflow model serving.
 
 Here is an example of a supervised job with EgoBipartiteSage, containing the following sections.
-1. Prepare data, including bulk-loading data for offline training and streaming data for online inference.
-2. Train the EgoBipartiteSage model using the offline bulk-loading data.
-3. Exporting TF model.
-4. Deploying TF model on tensorflow model serving.
+1. Prepare u2i and i2i bipartite graph data, including bulk-loading data for offline training and streaming data for online inference.
+2. Train the EgoBipartiteSage model using the offline bulk-loading data, it contains user model and item model.
+3. Exporting user model.
+4. Deploying the model on tensorflow model serving.
 5. Deploy the online dynamic graph service and ingest the streaming data.
 6. Start Java Client, sample and predict.
 
+The first 3 parts need to use the training framework GraphLearn-Training, a detailed description refs:[GraphLearn-Training](../gl/intro.md).
 
 ## 1. Prepare data
 
@@ -33,81 +34,53 @@ The generated data are stored in `/tmp/u2i_gen/training` and `/tmp/u2i_gen/strea
 
 ## 2. Train model offline
 
-FIXME(@Seventeen17): refactor this section
 ```shell
-cd graphlearn/examples/tf/ego_sage
-python train_supervised.py
+cd graphlearn/examples/tf/ego_bipartite_sage
+python train.py
 ```
 
 Ref to [GraphLearn-Training](../gl/intro.md) for more details.
 
 
 ## 3. Export TF SavedModel
 
-FIXME(@Seventeen17): refactor this section
 
 First, export model as tf SavedModel, we need to filter some of the placeholders as model serving inputs based on the
 computational graph, you can view the computational graph with the help of Tensorboard to determine the inputs for the
 serving subgraph.
 
-The offline training model is saved in `graphlearn/examples/tf/ego_sage/ckpt`, and we save the final serving model in
-`./ego_sage_sup_model` directory, the inputs to the subgraphs are the placeholders `0,2,3`.
+The offline training model is saved in `graphlearn/examples/tf/ego_bipartite_sage/ckpt`, and we save the final serving model in
+`graphlearn/examples/tf/serving/ego_bipartite_sage` directory, the inputs to the user-subgraph are the placeholders `0,3,4` according to TensorBoard.
 
 ```shell
 cd graphlearn/examples/tf/serving
-python export_serving_model.py ../ego_sage ckpt ego_sage_sup_model 0,2,3
+python export_serving_model.py --input_ckpt_dir=../ego_bipartite_sage --input_ckpt_name=ckpt --placeholders=0,3,4 --output_model_path=./ego_bipartite_sage
 ```
 
 Check inputs and output of saved model.
 ```shell
-saved_model_cli show --dir ego_sage_sup_model/1/ --all
+saved_model_cli show --dir ego_bipartite_sage/1/ --all
 ```
 
-Outputs are as following.
-```
-MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
-
-signature_def['predict_actions']:
-  The given SavedModel SignatureDef contains the following input(s):
-    inputs['IteratorGetNext_ph_input_0'] tensor_info:
-        dtype: DT_FLOAT
-        shape: (-1, 1433)
-        name: IteratorGetNext_placeholder:0
-    inputs['IteratorGetNext_ph_input_2'] tensor_info:
-        dtype: DT_FLOAT
-        shape: (-1, 1433)
-        name: IteratorGetNext_placeholder_2:0
-    inputs['IteratorGetNext_ph_input_3'] tensor_info:
-        dtype: DT_FLOAT
-        shape: (-1, 1433)
-        name: IteratorGetNext_placeholder_3:0
-  The given SavedModel SignatureDef contains the following output(s):
-    outputs['output_embeddings'] tensor_info:
-        dtype: DT_FLOAT
-        shape: (-1, 7)
-        name: output_embeddings:0
-  Method name is: tensorflow/serving/predict
-```
-
-
 ## 4. Deploy TF Model
-FIXME(@Seventeen17): refactor this section
 
 Install tensorflow-model-server.
+
 ```shell
 echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | sudo tee /etc/apt/sources.list.d/tensorflow-serving.list && \
 curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | sudo apt-key add -
 apt-get update && apt-get install tensorflow-model-server
 ```
+
 Start tensorflow-model-server and deploy model
+
 ```shell
 nohup tensorflow_model_server --port=9000  \
---model_name=saved_model_modified   \
---model_base_path=/home/wenting.swt/code/graph-learn/examples/tf/ego_sage/saved_model_modified \
+--model_name=egomodel   \
+--model_base_path=graphlearn/examples/tf/serving/ego_bipartite_sage \
 >server.log 2>&1
 ```
 
-
 ## 5. Deploy Dynamic Graph Service.
 
 ### 5.1 Deploy kafka service
@@ -191,39 +164,18 @@ cd dynamic_graph_service/dataloader
 
 
 ## 6. Sample and Predict
-We show a quick start example without dgs.
-TODO(@Seventeen17): replace me when dgs deployment doc is ready.
+
+We provide a Java Client for Sampling from DGS and predict with TF model service.
 
 ```
-cd dgs/gsl_client
-mvn -Dtest=PredictClientTest test
-```
+cd dynamic_graph_service/gsl_client
+mvn clean compile assembly:single
 
-The complete usgaes are shown in `App`.
-
-```java
-String server = "http://dynamic-graph-service.info";
-Graph g = Graph.connect(server);
-
-Query query = g.V("user").feed(source).properties(1).alias("seed")
-    .outV("u2i").sample(15).by("topk_by_timestamp").properties(1).alias("hop1")
-    .outV("i2i").sample(10).by("topk_by_timestamp").properties(1).alias("hop2")
-    .values();
-Status s = g.install(query);
-
-Decoder decoder = new Decoder(g);
-decoder.addFeatDesc("user",
-                    new ArrayList<String>(Arrays.asList("float")),
-                    new ArrayList<Integer>(Arrays.asList(128)));
-decoder.addFeatDesc("item",
-                    new ArrayList<String>(Arrays.asList("float")),
-                    new ArrayList<Integer>(Arrays.asList(128)));
-ArrayList<Integer> phs = new ArrayList<Integer>(Arrays.asList(0, 1, 2));
-
-TFPredictClient client = new TFPredictClient(decoder, "localhost", 9000);
-for (int i = 0; i < iters; ++i) {
-  Value content = g.run(query);
-  EgoGraph egoGraph = content.getEgoGraph("seed");
-  client.predict("model", 1, egoGraph, phs);
-}
+java -jar gsl_client-1.0-SNAPSHOT-jar-with-dependencies.jar http://dynamic-graph-service.info egomodel
 ```
+
+Java command with the following args,
+0: DGS service host name
+1: Serving model name.
+
+refs to `dynamic_graph_service/gsl_client/src/main/java/org/aliyun/App.java`

docs/en/dgs/deploy.md

@@ -10,29 +10,29 @@ Welcome to GraphLearn's documentation!
    :maxdepth: 2
    :caption: Introduction
 
-   intro
+   en/intro
 
 .. toctree::
    :maxdepth: 3
    :caption: GraphLearn-Training
 
-   gl/intro
-   gl/install
-   gl/quick_start
-   gl/graph/index
-   gl/algo/index
-   gl/developer/index
+   en/gl/intro
+   en/gl/install
+   en/gl/quick_start
+   en/gl/graph/index
+   en/gl/algo/index
+   en/gl/developer/index
 
 
 .. toctree::
    :maxdepth: 2
    :caption: Dynamic-Graph-Service
 
-   dgs/intro
-   dgs/deploy
-   dgs/dataloader
-   dgs/tutorial
-   dgs/developer
+   en/dgs/intro
+   en/dgs/deploy
+   en/dgs/dataloader
+   en/dgs/tutorial
+   en/dgs/developer
    apis/index
 
 

docs/en/dgs/tutorial.md

@@ -10,29 +10,29 @@ Welcome to GraphLearn's documentation!
    :maxdepth: 2
    :caption: Introduction
 
-   intro
+   zh_CN/intro
 
 .. toctree::
    :maxdepth: 3
    :caption: GraphLearn-Training
 
-   gl/intro
-   gl/install
-   gl/quick_start
-   gl/graph/index
-   gl/algo/index
-   gl/developer/index
+   zh_CN/gl/intro
+   zh_CN/gl/install
+   zh_CN/gl/quick_start
+   zh_CN/gl/graph/index
+   zh_CN/gl/algo/index
+   zh_CN/gl/developer/index
 
 
 .. toctree::
    :maxdepth: 2
    :caption: Dynamic-Graph-Service
 
-   dgs/intro
-   dgs/deploy
-   dgs/dataloader
-   dgs/tutorial
-   dgs/developer
+   zh_CN/dgs/intro
+   zh_CN/dgs/deploy
+   zh_CN/dgs/dataloader
+   zh_CN/dgs/tutorial
+   zh_CN/dgs/developer
    apis/index
 
 

docs/images/dgs_arch.png

@@ -1,6 +1,6 @@
 sphinx
 sphinx_rtd_theme
 recommonmark
-markdown==3.3.7
+markdown
 sphinx-markdown-tables
 myst-parser