updated abn sdk tutorial (#95)

* updated abn sdk tutorial

Signed-off-by: Michael Kalantar <[email protected]>

* fix spelling

Signed-off-by: Michael Kalantar <[email protected]>

* modify generate_load

Signed-off-by: Michael Kalantar <[email protected]>

* abn updates

Signed-off-by: Michael Kalantar <[email protected]>

* fix spelling, links

Signed-off-by: Michael Kalantar <[email protected]>

* reviwer comments, kustomize, python

Signed-off-by: Michael Kalantar <[email protected]>

* add dashboard image

Signed-off-by: Michael Kalantar <[email protected]>

* update documentation

Signed-off-by: Michael Kalantar <[email protected]>

---------

Signed-off-by: Michael Kalantar <[email protected]>

Author: kalantar

.github/wordlist.txt

@@ -23,6 +23,7 @@ frontend
 GitOps
 gRPC
 GitHub
+Grafana
 GVR
 Homebrew
 http
@@ -87,7 +88,6 @@ warmupNumRequests
 webhook
 webhooks
 yaml
-abnmetrics
 auth
 argoproj
 custommetrics
@@ -124,10 +124,18 @@ rollout
 rollouts
 setName
 setUser
-trackToRoute
+versionNumberToRoute
+GetVersionNumber
+getVersionnumber
+versionNumber
+versionRecommendation
 toJson
 versionname
 WriteMetric
+MetricValue
+setApplication
+setValue
+toString
 contentType
 numRequests
 payloadStr
@@ -159,4 +167,4 @@ wget
 gz
 xvf
 IMG
-mv
+mv

.github/workflows/linkcheck.yaml

@@ -21,7 +21,7 @@ jobs:
 
     - name: Link checker
       id: lychee
-      uses: lycheeverse/lychee-action@v1.6.1
+      uses: lycheeverse/lychee-action@v1.8.0
       env:
         GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
       with:

.lycheeignore

@@ -5,4 +5,8 @@ https://twitter.com/Iter8Tools
 
 https://raw.githubusercontent.com/*
 https://kserve.github.io/*
-https://docs.litmuschaos.io/docs/*
+https://docs.litmuschaos.io/docs/*
+
+http://localhost:3000
+http://localhost:3000/dashboards
+http://localhost:3000/connections/datasources/marcusolsson-json-datasource

docs/tutorials/abn/abn.md

@@ -2,173 +2,183 @@
 template: main.html
 ---
 
-# A/B Experiments
+# A/B Testing with the Iter8 SDK
 
-This tutorial describes an [A/B testing](../../user-guide/topics/ab_testing.md) experiment for a backend component.
+This tutorial describes how to do A/B testing of a backend component using the [Iter8 SDK](../../user-guide/topics/ab_testing.md). 
 
 ![A/B/n experiment](images/abn.png)
 
 ***
 
 ???+ warning "Before you begin"
     1. Try [your first experiment](../../getting-started/your-first-experiment.md). Understand the main [concepts](../../getting-started/concepts.md) behind Iter8 experiments.
+    2. Have Grafana available. For example, Grafana can be installed on your cluster as follows:
+    ```shell
+    kubectl create deploy grafana --image=grafana/grafana
+    kubectl expose deploy grafana --port=3000
+    ```
 
-## Launch Iter8 A/B/n service
-
-Deploy the Iter8 A/B/n service. When deploying the service, specify which Kubernetes resource types to watch for each application. To watch for versions of the *backend* application in the *default* namespace, configure the service to watch for Kubernetes service and deployment resources:
-
-```shell
-helm install --repo https://iter8-tools.github.io/iter8 iter8-abn abn \
---set "apps.default.backend.resources={service,deployment}"
-```
+## Launch the Iter8 A/B/n service
 
-??? warn "Assumptions"
-    To simplify specification, Iter8 assumes certain conventions:
-
-    - The baseline track identifier is the application name
-    - Track identifiers associated with candidate versions are of the form `<application_name>-candidate-<index>`
-    - All resource objects for all versions are deployed in the same namespace
-    - There is only 1 resource object of a given type in each version
-    - The name of each object in the version associated with the baseline track is the application name
-    - The name of each object in the version associated with a candidate track is of the form  `<application_name>-candidate-<index>` where index is 1, 2, etc.
+Deploy the Iter8 A/B/n service using either `helm` or `kustomize`:
 
+--8<-- "docs/tutorials/installiter8controller.md"
 
 ## Deploy the sample application
 
-Deploy both the frontend and backend components of the application as described in each tab:
+A sample application using the Iter8 SDK is provided. Deploy both the frontend and backend components of this application as described in each tab:
 
 === "frontend"
     Install the frontend component using an implementation in the language of your choice:
 
     === "node"
         ```shell
-        kubectl create deployment frontend --image=iter8/abn-sample-frontend-node:0.13
+        kubectl create deployment frontend --image=iter8/abn-sample-frontend-node:0.15.0
         kubectl expose deployment frontend --name=frontend --port=8090
         ```
+        <!-- kubectl create deployment frontend --image=kalantar/frontend-node:20230717-1552 -->
 
     === "Go"
         ```shell
-        kubectl create deployment frontend --image=iter8/abn-sample-frontend-go:0.13
+        kubectl create deployment frontend --image=iter8/abn-sample-frontend-go:0.15.0
         kubectl expose deployment frontend --name=frontend --port=8090
         ```
+        <!-- kubectl create deployment frontend --image=kalantar/frontend-go:20230717-1339 -->
 
-    The frontend component is implemented to call *Lookup()* before each call to the backend component. The frontend componet uses the returned track identifier to route the request to a version of the backend component.
+    The frontend component is implemented to call `Lookup()` before each call to the backend component. The frontend component uses the returned version number to route the request to the recommended version of the backend component.
 
 === "backend"
-    Deploy version *v1* of the *backend* component, associating it with the track identifier *backend*.
+    Deploy an initial version of the *backend* component:
 
     ```shell
     kubectl create deployment backend --image=iter8/abn-sample-backend:0.13-v1
-    kubectl label deployment backend app.kubernetes.io/version=v1
+    kubectl label deployment backend iter8.tools/watch="true"
 
     kubectl expose deployment backend --name=backend --port=8091
     ```
 
+## Describe the Application
+
+In order to support `Lookup()`, Iter8 needs to know what the application component versions look like. A `ConfigMap` is used to describe the make up of possible versions:
+
+```shell
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: backend
+  labels:
+    app.kubernetes.io/managed-by: iter8
+    iter8.tools/kind: routemap
+    iter8.tools/version: "v0.15"
+immutable: true
+data:
+  strSpec: |
+    versions:
+    - resources:
+      - gvrShort: svc
+        name: backend
+        namespace: default
+      - gvrShort: deploy
+        name: backend
+        namespace: default
+    - resources:
+      - gvrShort: svc
+        name: backend-candidate-1
+        namespace: default
+      - gvrShort: deploy
+        name: backend-candidate-1
+        namespace: default
+EOF
+```
+
+In this definition, each version of the application is composed of a `Service` and a `Deployment`. In the primary version, both are named `backend`. In any candidate version they are named `backend-candidate-1`. Iter8 uses this definition to identify when any of the versions of the application are available. It can then respond appropriate to `Lookup()` requests. 
+
 ## Generate load
 
-Generate load. In separate shells, port-forward requests to the frontend component and generate load for multiple users.  A [script](https://raw.githubusercontent.com/iter8-tools/docs/main/samples/abn-sample/generate_load.sh) is provided to do this. To use it:
+In separate shells, port-forward requests to the frontend component and generate load for multiple users. A [script](https://raw.githubusercontent.com/iter8-tools/docs/main/samples/abn-sample/generate_load.sh) is provided to do this. To use it:
     ```shell
     kubectl port-forward service/frontend 8090:8090
     ```
     ```shell
-    curl -s https://raw.githubusercontent.com/iter8-tools/docs/main/samples/abn-sample/generate_load.sh | sh -s --
+    curl -s https://raw.githubusercontent.com/iter8-tools/docs/v0.15.0/samples/abn-sample/generate_load.sh | sh -s --
     ```
+    <!-- # source /Users/kalantar/projects/go.workspace/src/github.com/iter8-tools/docs/samples/abn-sample/generate_load.sh -->
 
 ## Deploy a candidate version
 
-Deploy version *v2* of the *backend* component, associating it with the track identifier *backend-candidate-1*.
+Deploy the candidate version of the *backend* component, naming it `backend-candidate-1`.
 
 ```shell
 kubectl create deployment backend-candidate-1 --image=iter8/abn-sample-backend:0.13-v2
-kubectl label deployment backend-candidate-1 app.kubernetes.io/version=v2
+kubectl label deployment backend-candidate-1 iter8.tools/watch="true"
 
 kubectl expose deployment backend-candidate-1 --name=backend-candidate-1 --port=8091
 ```
 
-Until the candidate version is ready; that is, until all expected resources are deployed and available, calls to *Lookup()* will return only the *backend* track identifier.
-Once the candidate version is ready, *Lookup()* will return both track identifiers so that requests will be distributed between versions.
+Until the candidate version is ready; that is, until all expected resources are deployed and available, calls to `Lookup()` will return only the index 0; the existing version.
+Once the candidate version is ready, `Lookup()` will return both indices (0 and 1) so that requests can be distributed across versions.
+
+## Compare versions using Grafana
 
-## Launch experiment
+Inspect the metrics using Grafana. If Grafana is deployed to your cluster, port-forward requests as follows:
 
 ```shell
-iter8 k launch \
---set abnmetrics.application=default/backend \
---set "tasks={abnmetrics}" \
---set runner=cronjob \
---set cronjobSchedule="*/1 * * * *"
+kubectl port-forward service/grafana 3000:3000
 ```
 
-??? note "About this experiment"
-    This experiment periodically (in this case, once a minute) reads the `abn` metrics associated with the *backend* application component in the *default* namespace. These metrics are written by the frontend component using the *WriteMetric()* interface as a part of processing user requests.
-
-## Inspect experiment report
-
-Inspect the metrics:
+Open Grafana in a browser:
 
 ```shell
-iter8 k report
+http://localhost:3000/
 ```
 
-??? note "Sample output from report"
-    ```
-    Experiment summary:
-    *******************
-
-    Experiment completed: true
-    No task failures: true
-    Total number of tasks: 1
-    Number of completed tasks: 1
-    Number of completed loops: 3
-
-    Latest observed values for metrics:
-    ***********************************
-
-    Metric                   | backend (v1) | backend-candidate-1 (v2)
-    -------                  | -----        | -----
-    abn/sample_metric/count  | 35.00        | 28.00
-    abn/sample_metric/max    | 99.00        | 100.00
-    abn/sample_metric/mean   | 56.31        | 52.79
-    abn/sample_metric/min    | 0.00         | 1.00
-    abn/sample_metric/stddev | 28.52        | 31.91
-    ```
-The output allows you to compare the versions against each other and select a winner. Since the experiment runs periodically, the values in the report will change over time.
+[Add a JSON API data source](http://localhost:3000/connections/datasources/marcusolsson-json-datasource) `Iter8` with:
 
-Once a winner is identified, the experiment can be terminated, the winner can be promoted, and the candidate version(s) can be deleted.
+- URL `http://iter8.default:8080/metrics` and 
+- query string `application=default%2Fbackend`
 
-To delete the experiment:
+[Create a new dashboard](http://localhost:3000/dashboards) by *import*. Do so by pasting the contents of this [JSON definition](https://gist.githubusercontent.com/Alan-Cha/aa4ba259cc4631aafe9b43500502c60f/raw/034249f24e2c524ee4e326e860c06149ae7b2677/gistfile1.txt) into the box and *load* it. Associate it with the JSON API data source defined above.
 
-```shell
-iter8 k delete
-```
+The Iter8 dashboard allows you to compare the behavior of the two versions of the backend component against each other and select a winner. Since user requests are being sent by the load generation script, the values in the report may change over time.  The Iter8 dashboard may look like the following:
+
+![A/B dashboard](images/dashboard.png)
+
+Once a winner is identified, the winner can be promoted, and the candidate version deleted.
 
 ## Promote candidate version
 
-Delete the candidate version:
+To promote the candidate version (`backend-candidate-1`), first update the primary version, `backend`, using the new image. You can also overwrite any metadata describing the version.
 
 ```shell
-kubectl delete deployment backend-candidate-1 
-kubectl delete service backend-candidate-1
+kubectl set image deployment/backend abn-sample-backend=iter8/abn-sample-backend:0.13-v2
 ```
 
-Update the version associated with the baseline track identifier *backend*:
+Finally, delete the candidate version:
 
 ```shell
-kubectl set image deployment/backend abn-sample-backend=iter8/abn-sample-backend:0.13-v2
-kubectl label --overwrite deployment/backend app.kubernetes.io/version=v2
+kubectl delete svc/backend-candidate-1 deploy/backend-candidate-1
 ```
 
+Calls to `Lookup()` will now recommend that all traffic be sent to the primary version `backend` (currently serving the promoted version of the code).
+
 ## Cleanup
 
 ### Delete sample application
 
 ```shell
 kubectl delete \
-deploy/frontend deploy/backend deploy/backend-candidate-1 \
-service/frontend service/backend service/backend-candidate-1
+svc/frontend deploy/frontend \
+svc/backend deploy/backend \
+svc/backend-candidate-1 deploy/backend-candidate-1
 ```
 
-### Uninstall the A/B/n service
+### Delete the application description
 
 ```shell
-helm delete iter8-abn
+kubectl delete cm/backend
 ```
+
+### Uninstall the A/B/n service
+
+--8<-- "docs/tutorials/deleteiter8controller.md"