From f19d118510a96a2a86b45dcb15bf1ffce70b3a79 Mon Sep 17 00:00:00 2001 From: "Eli C. Lowry" <83078660+Enkidu93@users.noreply.github.com> Date: Thu, 5 Oct 2023 07:49:36 -0400 Subject: [PATCH] Reorganization of README and minor changes to formatting for ease of use and consistency (#149) After helping Matthew get Serval running, I've realized that although the README has a lot of information, it's not organized as helpfully as it could be. --- README.md | 101 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 60 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index 113b6ade..f467eedb 100644 --- a/README.md +++ b/README.md @@ -8,33 +8,40 @@ Serval is a REST API for natural language processing services. ## Development -### CSharpier - -All C# code should be formatted using [CSharpier](https://csharpier.com/). The best way to enable support for CSharpier is to install the appropriate [IDE extension](https://csharpier.com/docs/Editors) and configure it to format on save. - -### Coding conventions +### Development in Docker Compose -[Here](https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals/coding-style/identifier-names) is a good overview of naming conventions. [Here](https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals/coding-style/coding-conventions) is a good overview of coding conventions. If you want to get in to even more detail, check out the [Framework design guidelines](https://learn.microsoft.com/en-us/dotnet/standard/design-guidelines/). +This is the simplest way to develop. +First, git clone the repo: + ``` + git clone https://github.com/sillsdev/serval.git + ``` +Then build Serval: + ``` + dotnet build serval + ``` +Clone the [Machine repo](https://github.com/sillsdev/machine) into an adjacent folder to Serval: + ``` + git clone https://github.com/sillsdev/machine.git + ``` +Build the Machine repo: + ``` + dotnet build machine + ``` +Make sure that the environment variable ASPNETCORE_ENVIRONMENT is set to "Development" by running `export ASPNETCORE_ENVIRONMENT=Development` or adding it to your `.bashrc`. +In the Serval root, run docker compose up + ``` + cd serval && docker compose up + ``` +If using vscode, launch "DockerComb" to debug Serval and the for-testing-only Echo Engine. ### Development locally +Alternatively, you can develop without containerizing Serval. -- Install MongoDB 6.0 as a replca set and MongoDBCompass and run it on localhost:27017 -- set the following environment variables: - - ASPNETCORE_ENVIRONMENT=Development -- Open "Serval.sln" and debug the ApiServer -- Now, you are running the complete environment where everything is being debugged and the mongodb is exposed. +Install MongoDB 6.0 as a replica set run it on localhost:27017. (You can run `docker compose -f "docker-compose.mongo.yaml" up` from the root of the serval repo to do so). -#### local dev in ubuntu -* Make sure you install dotnet from the microsoft apr-get repo, otherwise debugging doesn't work! Also you will need to "searchMicrosoftSymbolServer" for the symbols. -* To have a local MongoDB, use the docker-compose script to create a replica set of one instance. +Make sure that the environment variable ASPNETCORE_ENVIRONMENT is set to "Development" by running `export ASPNETCORE_ENVIRONMENT=Development` or adding it to your `.bashrc`. -### Development in Docker Compose - -* Build this repository with dotnet build -* Download this repository and place the https://github.com/sillsdev/machine repo in ../machine, relative to this repo. -* Build the machine repo with dotnet build in the root of that repo -* In the serval root, run docker-compose up -* To debug serval and echo together, launch "ServalComb" in VSCode +Open "Serval.sln" and debug the ApiServer. ## Deployment on kubernetes There are 3 different environments that Serval is deployed to: @@ -52,7 +59,7 @@ There are 3 different environments that Serval is deployed to: - `helm install serval deploy/serval -n nlp -f deploy/qa-int-values.yaml` ### To update the cluster -- To upgrade serval: +- To upgrade Serval: - For QA internal Run: - `kubectl config use-context dallas-rke` - `helm upgrade serval deploy/serval -n nlp -f deploy/qa-int-values.yaml` @@ -63,8 +70,8 @@ There are 3 different environments that Serval is deployed to: - `kubectl config use-context aws-rke` - `helm upgrade serval deploy/serval -n serval -f deploy/values.yaml` +## Debugging ### To access Serval API - * Internal QA: * Use the VPN * In `C:\Windows\System32\drivers\etc\hosts`, enter in a line for `10.3.0.119 serval-api.org` @@ -72,29 +79,41 @@ There are 3 different environments that Serval is deployed to: * External QA: * go to `https://qa.serval-api.org/swagger` and accept the security warning -### Pod logs: - +### To view pod logs: - Run: `kubectl get pods` to get the currently running pods - Run: `kubectl logs ` - Run: `kubectl describe pod ` to check a stalled pod stuck in containercreating -### API E2E Testing -- Get auth0 token using curl: - - Get Client ID and Client Secret from auth0.com - - Login, go to Applications-> Applications -> "Machine API (Test Application)", or similar - - Copy `Client ID` into Environment variable `SERVAL_CLIENT_ID` - - Copy `Client Secret` into Environment variable `SERVAL_CLIENT_SECRET` - - Copy the auth0 url into Environment variable `SERVAL_AUTH_URL` (i.e. `SERVAL_AUTH_URL=https://sil-appbuilder.auth0.com`) - - Set `SERVAL_HOST_URL` to the api's URL (i.e. `SERVAL_HOST_URL=http://localhost`) - - Run tests from `Serval.E2ETests` - - The token will automatically be retrieved from Auth0 when you run the tests +### Running the API E2E Tests +In order to run the E2E tests, you will need to have the appropriate credentials +- Get Client ID and Client Secret from auth0.com + - Login, go to Applications-> Applications -> "Machine API (Test Application)" or similar + - Copy `Client ID` into Environment variable `SERVAL_CLIENT_ID` + - Copy `Client Secret` into Environment variable `SERVAL_CLIENT_SECRET` + - Copy the auth0 url into Environment variable `SERVAL_AUTH_URL` (e.g. `SERVAL_AUTH_URL=https://sil-appbuilder.auth0.com`) + - Set `SERVAL_HOST_URL` to the api's URL (e.g. `SERVAL_HOST_URL=http://localhost`) +Now, when you run the tests from `Serval.E2ETests`, the token will automatically be retrieved from Auth0. + +### Debugging the S3 bucket + +To view files stored in the bucket, run + ``` + aws s3 ls s3://aqua-ml-data/ + ``` +### Mongo debugging + +* First, get the mongo pod name: `kubectl get pods -n serval` +* Then forward to a local port, such as 28015: `kubectl port-forward 28015:27017 -n serval` +* Download [MongoDB Compass](https://www.mongodb.com/try/download/compass). +* Then, open MongoDB Compass and connect to `mongodb://localhost:28015/?directConnection=true` + -### Access S3 bucket +## Development Notes +### CSharpier -- to view files, run `aws s3 ls s3://aqua-ml-data/` +All C# code should be formatted using [CSharpier](https://csharpier.com/). The best way to enable support for CSharpier is to install the appropriate [IDE extension](https://csharpier.com/docs/Editors) and configure it to format on save. -### Mongo investigation +### Coding conventions + +[Here](https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals/coding-style/identifier-names) is a good overview of naming conventions. [Here](https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals/coding-style/coding-conventions) is a good overview of coding conventions. If you want to get in to even more detail, check out the [Framework design guidelines](https://learn.microsoft.com/en-us/dotnet/standard/design-guidelines/). -* First, get the pod name: `kubectl get pods -n serval` -* Then forward to local port, such as 28015: `kubectl port-forward 28015:27017 -n serval` -* Then open MongoDB Compass and connect to `mongodb://localhost:28015/?directConnection=true` \ No newline at end of file