Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global variable support and Generate env JSON option #24

Merged
merged 23 commits into from
Jul 20, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
cca4db6
search and load l2config.env file
lovestaco Jul 9, 2023
0c87717
consolidating LoadElfEnv and LoadConfigEnv
lovestaco Jul 9, 2023
84a4848
Adding comments for LoadEnvironments function
lovestaco Jul 10, 2023
4dd0f23
Updating documentation with global variable support
lovestaco Jul 10, 2023
655a236
examples.md
lovestaco Jul 12, 2023
548e43e
Updating preprocess, documentation
lovestaco Jul 12, 2023
48f3154
Updating architecture, documentation
lovestaco Jul 12, 2023
d12b69b
Updating l2format, documentation
lovestaco Jul 12, 2023
549affb
Adding command to get JSON of envs
lovestaco Jul 16, 2023
a66383b
Updating l2format and examples, documentation
lovestaco Jul 16, 2023
be15f49
Search l2configenv from the l2file level
lovestaco Jul 17, 2023
a1b1b33
Pure functions for GetL2EnvVariables
lovestaco Jul 18, 2023
caece11
Updating documentation
lovestaco Jul 18, 2023
36d2d4f
Make lint and gofumpt, lint errors
lovestaco Jul 18, 2023
ffc95e9
Adding varjson variable example, documentation
lovestaco Jul 18, 2023
82690f1
Adding root override and backtick echo var, test
lovestaco Jul 19, 2023
c19afee
Adding env_command, test
lovestaco Jul 20, 2023
ce1359c
Updating log to os.Stderr, logging
lovestaco Jul 20, 2023
251ddef
Linting and adding verbose, test
lovestaco Jul 20, 2023
cbe799f
Adding env_command, test
lovestaco Jul 20, 2023
58fbc92
Updating testapp.yml, build and test
lovestaco Jul 20, 2023
2b505fc
Relative path for binary, test
lovestaco Jul 20, 2023
312d5bd
Fixing all errors, linting
lovestaco Jul 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions controller/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ package contoller
import (
"fmt"
"os"
"path"

"github.com/HexmosTech/gabs/v2"
"github.com/HexmosTech/httpie-go"
Expand Down Expand Up @@ -88,7 +87,18 @@ func Process(version string) {
_, dir, _ := utils.GetFilePathComponents(o.Positional.LamaAPIFile)
oldDir, _ := os.Getwd()
utils.ChangeWorkingDir(dir)
preprocess.LoadElfEnv(path.Join(dir, "l2.env"))

if o.Env {
jsonEnvs,err := preprocess.GetL2EnvVariables(dir)
if err != nil {
log.Error().Str("Type", "Preprocess").Msg(err.Error())
return
}
log.Info().Str("Type", "Preprocess").Msg("Env Variables:\n" + string(jsonEnvs))
return
}

preprocess.LoadEnvironments(dir)
utils.ChangeWorkingDir(oldDir)
p := parser.NewLama2Parser()
parsedAPI, e := p.Parse(apiContent)
Expand Down
40 changes: 29 additions & 11 deletions docs/Lama2/docs/explanation/l2format.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ the rules for authoring `.l2` API files. This
document expects some familiarity with *Lama2*.

To quickly get started with *Lama2*, head over
to [Examples](../tutorials/examples.md).
to [Examples](../tutorials/examples.md).

On the
other hand, if you are a developer and wish to
learn more about the formal grammar underlying
*l2*, visit the [Grammar](../reference/grammar.md)
_l2_, visit the [Grammar](../reference/grammar.md)
section.

### Comments start with `#`
Expand All @@ -21,7 +21,6 @@ Fully supported: `GET|HEAD|POST|PUT|DELETE|CONNECT|OPTIONS|TRACE|PATCH`

### JSON is the default submission type, but MULTIPART is supported too


#### `varjson` is a simpler syntax to specify flat JSONs

`varjson` values are defined as follows:
Expand All @@ -42,10 +41,10 @@ The above results in a JSON submission of the form:

#### Nested JSON can simply be dumped at the end of the document

The JSON recognition engine is quite lenient. It can deal with
The JSON recognition engine is quite lenient. It can deal with
minor errors in the format (such as having single quotes instead
of double quotes, trailing garbage, or an extra comma after the
last element in an array,).
last element in an array,).

```
POST
Expand All @@ -57,8 +56,6 @@ https://httpbin.org/post
}
```



#### MULTIPART allows both file uploads & the usual fields

Example:
Expand All @@ -75,15 +72,15 @@ file@./helloworld.jpg

The *file path is relative to the request file.*

### Cookies are sent as headers
### Cookies are sent as headers

Cookies are specified in a `Cookie` header as follows:

```
Cookie:'sessionid=foo;another-cookie=bar'
```

### Environments variables/commands can be defined in `<requests_dir>/l2.env`
### API environment variables can be defined locally in `l2.env`

By default, *l2* looks for a `l2.env` file in the same directory as the given
request file directory. Example `l2.env`:
Expand All @@ -93,6 +90,27 @@ export PHOTO=`base64 aadhaarlarge.jpg`
export AHOST="http://localhost:8000"
```

Get [Source Files](https://github.com/HexmosTech/Lama2/tree/main/examples/0004_env_switch_root)

### API environment variables can be defined at project level using `l2config.env`

_l2_ loads local variables from `l2.env`.
Copy link
Contributor

@shrsv shrsv Jul 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is confusing to the end user. You mean - at file level or at variable?

The fact is, we load variables from both files. If there is ever a conflict in a particular variable name, the way prioritize selection is:

  1. VM vars
  2. l2.env vars
  3. l2config.env vars

If not found then uses variables from `l2config.env`.
Example `l2config.env`:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot needed to show project structure, and root.

```
export PHOTO=`base64 aadhaarsmall.jpg`
export AHOST="http://localhost:8001"
```

Get [Source Files](https://github.com/HexmosTech/Lama2/tree/main/examples/0019_env_switch_global_root)

### If API environment variables are locally and at project level

The local variable's value is taken into consideration

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must show full example here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Get [Source Files](https://github.com/HexmosTech/Lama2/tree/main/examples/0020_override_project_root_local)

#### The environment file can load results of commands

Use the backtick notation `\`command\`` to place the results of
Expand All @@ -106,7 +124,7 @@ One can load the `PHOTO` variable in API files.

### Chain requests through Javascript blocks

*Lama2* supports plain Javascript (JS) blocks
_Lama2_ supports plain Javascript (JS) blocks
as a glue for manipulating responses and passing on
values to later stages. At a higher
level, a chain of requests may look like:
Expand All @@ -126,4 +144,4 @@ variable `result` contains the response from previous stages.

For example, in the above case, `Javascript 2` can access the response from `L2 Request 1` through the `result` variable.

Learn more about request chaining in [Examples](../tutorials/examples.md#chain-requests-using-javascript).
Learn more about request chaining in [Examples](../tutorials/examples.md#chain-requests-using-javascript).
5 changes: 3 additions & 2 deletions docs/Lama2/docs/reference/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
graph TD
K["Controller Entry <br/>(controller)"]
A["Parse CLI <br/>(lama2cmd)"]
B["Parser <br/>(parser)"]
B["Parser <br/>(parser)"]
D["Request Executor <br/>(cmdexec)"]
E["Output Format Manager <br/>(outputmanager)"]
F["Error Reporting (TODO)"]
Expand Down Expand Up @@ -48,7 +48,8 @@ From a high level, how does it work now?
2. Else if block is Requestor block
1. Replace variables with values in the following order
1. Try fetch variable from Javascript VM
2. If (1) fails, try fetch variable from `l2.env`
2. If (1) fails, try fetch Local env variable from `l2.env`
3. Try fetch Project env variable from `l2config.env`
2. Use the processed elements to create an httpie-go request
3. Fetch response
5. If necessary, write the last transaction to `.json` file
49 changes: 24 additions & 25 deletions docs/Lama2/docs/reference/preprocess.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,77 +10,76 @@ Package preprocess provides facilities to expand environment variables in \`.l2\

## Index

- [func Expand(s string, vm *goja.Runtime, mapping map[string]string) string](<#func-expand>)
- [func ExpandEnv(s string, vm *goja.Runtime) string](<#func-expandenv>)
- [func ExpandHeaders(block *gabs.Container, vm *goja.Runtime)](<#func-expandheaders>)
- [func ExpandJSON(block *gabs.Container, vm *goja.Runtime)](<#func-expandjson>)
- [func ExpandURL(block *gabs.Container, vm *goja.Runtime)](<#func-expandurl>)
- [func GetLamaFileAsString(path string) string](<#func-getlamafileasstring>)
- [func LamaFile(inputFile string) (string, string)](<#func-lamafile>)
- [func LoadElfEnv(l2path string)](<#func-loadelfenv>)
- [func ProcessVarsInBlock(block *gabs.Container, vm *goja.Runtime)](<#func-processvarsinblock>)


## func [Expand](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/expandvar.go#L19>)
- [func Expand(s string, vm \*goja.Runtime, mapping map[string]string) string](#func-expand)
- [func ExpandEnv(s string, vm \*goja.Runtime) string](#func-expandenv)
- [func ExpandHeaders(block *gabs.Container, vm *goja.Runtime)](#func-expandheaders)
- [func ExpandJSON(block *gabs.Container, vm *goja.Runtime)](#func-expandjson)
- [func ExpandURL(block *gabs.Container, vm *goja.Runtime)](#func-expandurl)
- [func GetLamaFileAsString(path string) string](#func-getlamafileasstring)
- [func LamaFile(inputFile string) (string, string)](#func-lamafile)
- [func LoadEnvFile(l2path string)](#func-loadenvfile)
- [func ProcessVarsInBlock(block *gabs.Container, vm *goja.Runtime)](#func-processvarsinblock)

## func [Expand](https://github.com/HexmosTech/Lama2/blob/master/preprocess/expandvar.go#L19)

```go
func Expand(s string, vm *goja.Runtime, mapping map[string]string) string
```

Expand replaces $\{var\} or $var in the string based on the mapping function. For example, os.ExpandEnv\(s\) is equivalent to os.Expand\(s, os.Getenv\).

## func [ExpandEnv](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/expandvar.go#L74>)
## func [ExpandEnv](https://github.com/HexmosTech/Lama2/blob/master/preprocess/expandvar.go#L74)

```go
func ExpandEnv(s string, vm *goja.Runtime) string
```

ExpandEnv replaces $\{var\} or $var in the string according to the values of the current environment variables. References to undefined variables are replaced by the empty string.

## func [ExpandHeaders](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L25>)
## func [ExpandHeaders](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L25)

```go
func ExpandHeaders(block *gabs.Container, vm *goja.Runtime)
```

## func [ExpandJSON](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L77>)
## func [ExpandJSON](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L77)

```go
func ExpandJSON(block *gabs.Container, vm *goja.Runtime)
```

## func [ExpandURL](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L45>)
## func [ExpandURL](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L45)

```go
func ExpandURL(block *gabs.Container, vm *goja.Runtime)
```

## func [GetLamaFileAsString](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L104>)
## func [GetLamaFileAsString](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L104)

```go
func GetLamaFileAsString(path string) string
```

## func [LamaFile](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L118>)
## func [LamaFile](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L118)

```go
func LamaFile(inputFile string) (string, string)
```

LamaFile takes in a path to an API file. It moves into the API file directory, reads the API contents, loads the \`l2.env\` file if available, and finally substitutes environment vars in the API contents Once done, it reverts back to the original directory, and returns the processed l2 file.
LamaFile takes in a path to an API file. It moves into the API file directory, reads the API contents, loads the \`l2config.env\` global variables file from the project directory.

Then loads the \`l2.env\` local variables file from the present directory, Variables which are already declared in \`l2config.env\` will be overwritten with local variable of \`l2.env\`. Once done, it reverts back to the original directory, and returns the processed l2 file.

## func [LoadElfEnv](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L97>)
## func [LoadEnvFile](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L97)

```go
func LoadElfEnv(l2path string)
func LoadEnvFile(l2path string)
```

## func [ProcessVarsInBlock](<https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L19>)
## func [ProcessVarsInBlock](https://github.com/HexmosTech/Lama2/blob/master/preprocess/preprocess.go#L19)

```go
func ProcessVarsInBlock(block *gabs.Container, vm *goja.Runtime)
```



Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)
Generated by [gomarkdoc](https://github.com/princjef/gomarkdoc)
Loading