Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

first attempt for knuu API #601

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,6 @@ coverage.txt
.devcontainer
Taskfile.yaml

bin/*
docker-compose.yml
.env
10 changes: 10 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@ pkgs := $(shell go list ./...)
run := .
count := 1
timeout := 120m
BINARY_NAME := knuu

build:
go build -o bin/$(BINARY_NAME) -v ./cmd

# docker:
# docker build -t $(BINARY_NAME) .

run: build
./bin/$(BINARY_NAME) api -l debug

## help: Show this help message
help: Makefile
Expand Down
208 changes: 208 additions & 0 deletions cmd/api/api.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
package api

import (
"context"
"fmt"

"github.com/gin-gonic/gin"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"gorm.io/gorm/logger"

"github.com/celestiaorg/knuu/internal/api/v1"
"github.com/celestiaorg/knuu/internal/api/v1/services"
"github.com/celestiaorg/knuu/internal/database"
)

const (
apiCmdName = "api"

flagPort = "port"
flagAPILogLevel = "log-level"

flagDBHost = "db.host"
flagDBUser = "db.user"
flagDBPassword = "db.password"
flagDBName = "db.name"
flagDBPort = "db.port"

flagSecretKey = "secret-key"
flagAdminUser = "admin-user"
flagAdminPass = "admin-pass"

flagTestsLogsPath = "tests-logs-path"

defaultPort = 8080
defaultLogLevel = gin.ReleaseMode

defaultDBHost = database.DefaultHost
defaultDBUser = database.DefaultUser
defaultDBPassword = database.DefaultPassword
defaultDBName = database.DefaultDBName
defaultDBPort = database.DefaultPort

defaultSecretKey = "secret"
defaultAdminUser = "admin"
defaultAdminPass = "admin"
Comment on lines +49 to +51
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Security: Replace insecure default values.

The default values for sensitive data are insecure:

  • Default secret key is too simple
  • Default admin credentials are too weak

Consider:

  1. Requiring these values to be provided through environment variables
  2. Adding validation for minimum password complexity
  3. Generating a secure random secret key if not provided

Example implementation:

-    defaultSecretKey = "secret"
-    defaultAdminUser = "admin"
-    defaultAdminPass = "admin"
+    defaultSecretKey = "" // Must be provided
+    defaultAdminUser = "" // Must be provided
+    defaultAdminPass = "" // Must be provided
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
defaultSecretKey = "secret"
defaultAdminUser = "admin"
defaultAdminPass = "admin"
defaultSecretKey = "" // Must be provided
defaultAdminUser = "" // Must be provided
defaultAdminPass = "" // Must be provided


defaultLogsPath = services.DefaultTestLogsPath
)

func NewAPICmd() *cobra.Command {
apiCmd := &cobra.Command{
Use: apiCmdName,
Short: "Start the Knuu API server",
Long: "Start the API server to manage tests, tokens, and users.",
RunE: runAPIServer,
}

apiCmd.Flags().IntP(flagPort, "p", defaultPort, "Port to run the API server on")
apiCmd.Flags().StringP(flagAPILogLevel, "l", defaultLogLevel, "Log level: debug | release | test")

apiCmd.Flags().StringP(flagDBHost, "d", defaultDBHost, "Postgres database host")
apiCmd.Flags().StringP(flagDBUser, "", defaultDBUser, "Postgres database user")
apiCmd.Flags().StringP(flagDBPassword, "", defaultDBPassword, "Postgres database password")
apiCmd.Flags().StringP(flagDBName, "", defaultDBName, "Postgres database name")
apiCmd.Flags().IntP(flagDBPort, "", defaultDBPort, "Postgres database port")

apiCmd.Flags().StringP(flagSecretKey, "", defaultSecretKey, "JWT secret key")
apiCmd.Flags().StringP(flagAdminUser, "", defaultAdminUser, "Admin username")
apiCmd.Flags().StringP(flagAdminPass, "", defaultAdminPass, "Admin password")

apiCmd.Flags().StringP(flagTestsLogsPath, "", defaultLogsPath, "Directory to store logs of the tests")

return apiCmd
}

func runAPIServer(cmd *cobra.Command, args []string) error {
dbOpts, err := getDBOptions(cmd.Flags())
if err != nil {
return fmt.Errorf("failed to get database options: %v", err)
}

db, err := database.New(dbOpts)
if err != nil {
return fmt.Errorf("failed to connect to database: %v", err)
}

apiOpts, err := getAPIOptions(cmd.Flags())
if err != nil {
return fmt.Errorf("failed to get API options: %v", err)
}

apiServer, err := api.New(context.Background(), db, apiOpts)
if err != nil {
return fmt.Errorf("failed to create API server: %v", err)
}

return apiServer.Start()
}
mojtaba-esk marked this conversation as resolved.
Show resolved Hide resolved

func getDBOptions(flags *pflag.FlagSet) (database.Options, error) {
dbHost, err := flags.GetString(flagDBHost)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get database host: %v", err)
}

dbUser, err := flags.GetString(flagDBUser)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get database user: %v", err)
}

dbPassword, err := flags.GetString(flagDBPassword)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get database password: %v", err)
}

dbName, err := flags.GetString(flagDBName)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get database name: %v", err)
}

dbPort, err := flags.GetInt(flagDBPort)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get database port: %v", err)
}

apiLogLevel, err := flags.GetString(flagAPILogLevel)
if err != nil {
return database.Options{}, fmt.Errorf("failed to get API log level: %v", err)
}

var dbLogLevel logger.LogLevel
switch apiLogLevel {
case gin.DebugMode:
dbLogLevel = logger.Info
case gin.ReleaseMode:
dbLogLevel = logger.Error
case gin.TestMode:
dbLogLevel = logger.Info
}

return database.Options{
Host: dbHost,
User: dbUser,
Password: dbPassword,
DBName: dbName,
Port: dbPort,
LogLevel: dbLogLevel,
}, nil
}

func getAPIOptions(flags *pflag.FlagSet) (api.Options, error) {
port, err := flags.GetInt(flagPort)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get port: %v", err)
}

apiLogLevel, err := flags.GetString(flagAPILogLevel)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get log level: %v", err)
}

secretKey, err := flags.GetString(flagSecretKey)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get secret key: %v", err)
}

adminUser, err := flags.GetString(flagAdminUser)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get admin user: %v", err)
}

adminPass, err := flags.GetString(flagAdminPass)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get admin password: %v", err)
}

testsLogsPath, err := flags.GetString(flagTestsLogsPath)
if err != nil {
return api.Options{}, fmt.Errorf("failed to get tests logs path: %v", err)
}

logger := logrus.New()
logger.SetFormatter(&logrus.JSONFormatter{})

switch apiLogLevel {
case gin.DebugMode:
logger.SetLevel(logrus.DebugLevel)
case gin.ReleaseMode:
logger.SetLevel(logrus.ErrorLevel)
case gin.TestMode:
logger.SetLevel(logrus.InfoLevel)
}

return api.Options{
Port: port,
APILogMode: apiLogLevel, // gin logger (HTTP request level)
SecretKey: secretKey,
AdminUser: adminUser,
AdminPass: adminPass,
Logger: logger, // handler (application level logger)
TestServiceOptions: services.TestServiceOptions{
TestsLogsPath: testsLogsPath, // directory to store logs of each test
Logger: logger,
},
}, nil
}
13 changes: 13 additions & 0 deletions cmd/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package main

import (
"github.com/sirupsen/logrus"

"github.com/celestiaorg/knuu/cmd/root"
)

func main() {
if err := root.Execute(); err != nil {
logrus.WithError(err).Fatal("failed to execute command")
}
}
22 changes: 22 additions & 0 deletions cmd/root/root.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package root

import (
"github.com/celestiaorg/knuu/cmd/api"

"github.com/spf13/cobra"
)

var rootCmd = &cobra.Command{
Use: "knuu",
Short: "Knuu CLI",
Long: "Knuu CLI provides commands to manage the Knuu API server and its operations.",
}

// Execute runs the root command.
func Execute() error {
return rootCmd.Execute()
}

func init() {
rootCmd.AddCommand(api.NewAPICmd())
}
64 changes: 32 additions & 32 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,39 +4,48 @@ go 1.22.5

require (
github.com/celestiaorg/bittwister v0.0.0-20231213180407-65cdbaf5b8c7
github.com/docker/docker v26.1.4+incompatible
github.com/gin-gonic/gin v1.10.0
github.com/golang-jwt/jwt v3.2.2+incompatible
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider upgrading to golang-jwt/jwt/v5.

The current JWT library version (v3.2.2+incompatible) is outdated. Version 5 provides better security features and type safety.

-	github.com/golang-jwt/jwt v3.2.2+incompatible
+	github.com/golang-jwt/jwt/v5 v5.2.0
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
github.com/golang-jwt/jwt v3.2.2+incompatible
github.com/golang-jwt/jwt/v5 v5.2.0

github.com/google/uuid v1.6.0
github.com/lib/pq v1.10.9
github.com/minio/minio-go/v7 v7.0.74
github.com/rs/cors v1.11.1
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.9.0
golang.org/x/crypto v0.29.0
gopkg.in/yaml.v2 v2.4.0
gorm.io/driver/postgres v1.5.11
gorm.io/gorm v1.25.12
k8s.io/api v0.30.2
k8s.io/apimachinery v0.30.2
k8s.io/client-go v0.30.2
k8s.io/utils v0.0.0-20230726121419-3b25d923346b
)

require (
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/acroca/go-symbols v0.1.1 // indirect
github.com/bytedance/sonic v1.11.6 // indirect
github.com/bytedance/sonic/loader v0.1.1 // indirect
github.com/cilium/ebpf v0.12.3 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/cloudwego/base64x v0.1.4 // indirect
github.com/cloudwego/iasm v0.2.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/distribution/reference v0.5.0 // indirect
github.com/docker/go-connections v0.4.1-0.20210727194412-58542c764a11 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/frankban/quicktest v1.14.6 // indirect
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
Expand All @@ -47,42 +56,37 @@ require (
github.com/gorilla/mux v1.8.1 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/pgx/v5 v5.5.5 // indirect
github.com/jackc/puddle/v2 v2.2.1 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/karrick/godirwalk v1.17.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/klauspost/cpuid/v2 v2.2.8 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/spdystream v0.2.0 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/nsf/gocode v0.0.0-20230322162601-b672b49f3818 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2.0.20221005185240-3a7f492d3f1b // indirect
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/ramya-rao-a/go-outline v0.0.0-20210608161538-9736a4bde949 // indirect
github.com/rs/xid v1.5.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/uudashr/gopkgs v1.3.2 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0 // indirect
go.opentelemetry.io/otel v1.26.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.26.0 // indirect
go.opentelemetry.io/otel/metric v1.26.0 // indirect
go.opentelemetry.io/otel/sdk v1.26.0 // indirect
go.opentelemetry.io/otel/trace v1.26.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.29.0 // indirect
golang.org/x/arch v0.8.0 // indirect
golang.org/x/exp v0.0.0-20240213143201-ec583247a57a // indirect
golang.org/x/mod v0.22.0 // indirect
golang.org/x/net v0.31.0 // indirect
golang.org/x/oauth2 v0.17.0 // indirect
golang.org/x/sync v0.9.0 // indirect
Expand All @@ -91,14 +95,10 @@ require (
golang.org/x/text v0.20.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.27.0 // indirect
golang.org/x/tools/cmd/guru v0.1.1-deprecated // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240227224415-6ceb2ff114de // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda // indirect
google.golang.org/protobuf v1.33.0 // indirect
google.golang.org/protobuf v1.34.1 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gotest.tools/v3 v3.5.1 // indirect
k8s.io/klog/v2 v2.120.1 // indirect
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
Expand Down
Loading
Loading