Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global variable support and Generate env JSON option #24

Merged
merged 23 commits into from
Jul 20, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
cca4db6
search and load l2config.env file
lovestaco Jul 9, 2023
0c87717
consolidating LoadElfEnv and LoadConfigEnv
lovestaco Jul 9, 2023
84a4848
Adding comments for LoadEnvironments function
lovestaco Jul 10, 2023
4dd0f23
Updating documentation with global variable support
lovestaco Jul 10, 2023
655a236
examples.md
lovestaco Jul 12, 2023
548e43e
Updating preprocess, documentation
lovestaco Jul 12, 2023
48f3154
Updating architecture, documentation
lovestaco Jul 12, 2023
d12b69b
Updating l2format, documentation
lovestaco Jul 12, 2023
549affb
Adding command to get JSON of envs
lovestaco Jul 16, 2023
a66383b
Updating l2format and examples, documentation
lovestaco Jul 16, 2023
be15f49
Search l2configenv from the l2file level
lovestaco Jul 17, 2023
a1b1b33
Pure functions for GetL2EnvVariables
lovestaco Jul 18, 2023
caece11
Updating documentation
lovestaco Jul 18, 2023
36d2d4f
Make lint and gofumpt, lint errors
lovestaco Jul 18, 2023
ffc95e9
Adding varjson variable example, documentation
lovestaco Jul 18, 2023
82690f1
Adding root override and backtick echo var, test
lovestaco Jul 19, 2023
c19afee
Adding env_command, test
lovestaco Jul 20, 2023
ce1359c
Updating log to os.Stderr, logging
lovestaco Jul 20, 2023
251ddef
Linting and adding verbose, test
lovestaco Jul 20, 2023
cbe799f
Adding env_command, test
lovestaco Jul 20, 2023
58fbc92
Updating testapp.yml, build and test
lovestaco Jul 20, 2023
2b505fc
Relative path for binary, test
lovestaco Jul 20, 2023
312d5bd
Fixing all errors, linting
lovestaco Jul 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions controller/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ package contoller
import (
"fmt"
"os"
"path"

"github.com/HexmosTech/gabs/v2"
"github.com/HexmosTech/httpie-go"
Expand Down Expand Up @@ -88,7 +87,7 @@ func Process(version string) {
_, dir, _ := utils.GetFilePathComponents(o.Positional.LamaAPIFile)
oldDir, _ := os.Getwd()
utils.ChangeWorkingDir(dir)
preprocess.LoadElfEnv(path.Join(dir, "l2.env"))
preprocess.LoadEnvironments(dir)
utils.ChangeWorkingDir(oldDir)
p := parser.NewLama2Parser()
parsedAPI, e := p.Parse(apiContent)
Expand Down
11 changes: 11 additions & 0 deletions docs/Lama2/docs/explanation/l2format.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,17 @@ Cookies are specified in a `Cookie` header as follows:
Cookie:'sessionid=foo;another-cookie=bar'
```

### Global Environments variables/commands can be defined in `/l2config.env/<requests_dir>`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of calling it "global", I think we should call it something like:

  1. Project-level variables
  2. Root variables
  3. Project-root variables

Let's pick one of the above and finalize (or make other suggestions).

Second, I think we should use git repo analogy to explain.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the git repo analogy in example.md, should I link it here as well?

Copy link
Contributor

@shrsv shrsv Jul 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Analogy should be in the explanation section, not example section. When confused what belongs where, re-read this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global Environments variables/commands can be defined in /l2config.env/<requests_dir>

Title is a bit clumsy.

API environment variables can be defined at project level using l2config.env

Is much simpler.


The *l2* loads up variables from `l2config.env` and then `l2.env`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again - the overwrite explanation is more complex to the reader/user. Simply say what order in which variables are searched for (nearest first):

  1. VM
  2. local
  3. global

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either:

The l2 binary loads...

or

l2 loads

If same variable is found in `l2.env` then overwrites the global variable.
Example `l2config.env`:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot needed to show project structure, and root.

```
export PHOTO=`base64 aadhaarsmall.jpg`
export AHOST="http://localhost:8001"
```

### Environments variables/commands can be defined in `<requests_dir>/l2.env`

By default, *l2* looks for a `l2.env` file in the same directory as the given
Expand Down
4 changes: 2 additions & 2 deletions docs/Lama2/docs/reference/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
graph TD
K["Controller Entry <br/>(controller)"]
A["Parse CLI <br/>(lama2cmd)"]
B["Parser <br/>(parser)"]
B["Parser <br/>(parser)"]
D["Request Executor <br/>(cmdexec)"]
E["Output Format Manager <br/>(outputmanager)"]
F["Error Reporting (TODO)"]
Expand Down Expand Up @@ -48,7 +48,7 @@ From a high level, how does it work now?
2. Else if block is Requestor block
1. Replace variables with values in the following order
1. Try fetch variable from Javascript VM
2. If (1) fails, try fetch variable from `l2.env`
2. If (1) fails, try fetch variable from `l2config.env` and `l2.env`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"overwriting" is an implementation detail. When you talk to users - how does it work?

  1. JS VM variable
  2. Local env variable
  3. l2config env variable.

Always explain in this way. The implementation detail doesn't matter at documentation/explanation level.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay got it

2. Use the processed elements to create an httpie-go request
3. Fetch response
5. If necessary, write the last transaction to `.json` file
2 changes: 1 addition & 1 deletion docs/Lama2/docs/reference/preprocess.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ func GetLamaFileAsString(path string) string
func LamaFile(inputFile string) (string, string)
```

LamaFile takes in a path to an API file. It moves into the API file directory, reads the API contents, loads the \`l2.env\` file if available, and finally substitutes environment vars in the API contents Once done, it reverts back to the original directory, and returns the processed l2 file.
LamaFile takes in a path to an API file. It moves into the API file directory, reads the API contents, loads the \`l2config.env\` global variables file if available in any of parent directory and \`l2.env\` file if available in present directory , and finally substitutes environment vars in the API contents Once done, it reverts back to the original directory, and returns the processed l2 file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment can make the "overwriting" part more obvious. Sentence is too long now, maybe simplify a bit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes


## func [LoadElfEnv](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L97>)

Expand Down
14 changes: 12 additions & 2 deletions docs/Lama2/docs/tutorials/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,18 @@ Get [Source File](https://github.com/HexmosTech/Lama2/tree/main/examples/0003_co

## Environment Variables: Switch base URL

Copy link
Contributor

@shrsv shrsv Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make the documentation more structured (something like this)

Case 1: l2.env adjacent to an API file

For any given .l2 file, one can place an l2.env file to store relevant variables. These variables will be available to be used within the API file

[example]

Case 2: Project-level variables

In Lama2, you can have a large number of API files stored in a hierarchical folder configuration. The root of such a project can be signified through l2config.env:

[API Hub folder screenshot with l2config.env highlighted]

Within such a structure, you can have an API file anywhere, which can use variables defined in the project-level variables:

[Example l2config.env and api.l2 with appropriate folder structure; the example must be hosted in Lama2 repo (see other examples)]

Case 3: Override Project-level variable with local variable

....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it

Specify variables in `l2.env` and then load
them up in the API files. Presently, **the `l2.env` file should reside in the same directory as the `.l2` API file.**
Specify global variables in `l2config.env` in parent directory.
**The global variables of `l2config.env` will be overwritten by local `l2.env` variables.**

Specify variables in `l2.env` and then load them up in the API files.
Presently, **the `l2.env` file should reside in the same directory as the `.l2` API file.**

**l2config.env**

```
export LOCAL="http://localhost:8001"
export REMOTE="http://httpbin.in"
```

**l2.env**

Expand Down
24 changes: 24 additions & 0 deletions preprocess/preprocess.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ import (
"fmt"
"io/ioutil"
"os"
"path"
"path/filepath"
"strings"

"github.com/HexmosTech/gabs/v2"
Expand Down Expand Up @@ -103,6 +105,28 @@ func LoadElfEnv(l2path string) {
}
}

func LoadConfigEnv(l2ConfigPath string) {
err := godotenv.Load(l2ConfigPath)
if err != nil {
parentDir := filepath.Dir(l2ConfigPath)
for parentDir != string(filepath.Separator) {
l2ConfigPath = filepath.Join(parentDir, "l2config.env")
err = godotenv.Load(l2ConfigPath)
if err == nil {
log.Info().Str("Type", "Preprocess").Str("l2config.env found in", parentDir).Msg("")
return // Found the l2config.env file, exit the function
}
parentDir = filepath.Dir(parentDir)
}
log.Error().Str("Type", "Preprocess").Msg("Didn't find l2config.env in the API directory")
}
}

func LoadEnvironments(dir string) {
LoadConfigEnv(path.Join(dir, "l2config.env")) // Loads global variables from l2config.env
LoadElfEnv(path.Join(dir, "l2.env")) // Overwrites the global variables if declared again in l2.env
}

func GetLamaFileAsString(path string) string {
b, err := ioutil.ReadFile(path) // just pass the file name
if err != nil {
Expand Down