Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add telemetry #144

Merged
merged 22 commits into from
Mar 26, 2025
Merged

add telemetry #144

merged 22 commits into from
Mar 26, 2025

Conversation

juev
Copy link
Member

@juev juev commented Mar 7, 2025

fixed: #58

@juev juev requested review from yalosev and ldmonster March 7, 2025 13:19
@juev juev self-assigned this Mar 7, 2025
@juev juev marked this pull request as ready for review March 7, 2025 13:23
ldmonster
ldmonster reviewed Mar 11, 2025
View reviewed changes
@juev juev marked this pull request as draft March 24, 2025 09:26
juev added 2 commits March 25, 2025 08:59
@juev
feat: add metrics file support and telemetry tracking for lint rule e…
7056a23 
…rrors

diff --git c/internal/flags/flags.go i/internal/flags/flags.go
index 5abec33..fc43c3b 100644
--- c/internal/flags/flags.go
+++ i/internal/flags/flags.go
@@ -34,6 +34,7 @@ var (
 	PrintVersion bool
 	Version      string
 	ValuesFile   string
+	MetricsFile  string
 )

 func InitDefaultFlagSet() *pflag.FlagSet {
@@ -52,6 +53,7 @@ func InitLintFlagSet() *pflag.FlagSet {
 	lint.StringVar(&LinterName, "linter", "", "linter name to run")
 	lint.StringVarP(&LogLevel, "log-level", "l", "INFO", "log-level [DEBUG | INFO | WARN | ERROR]")
 	lint.StringVarP(&ValuesFile, "values-file", "f", "", "path to values.yaml file with override values")
+	lint.StringVarP(&MetricsFile, "metrics-file", "m", "", "path to metrics.txt file")

 	return lint
 }
diff --git c/internal/manager/manager.go i/internal/manager/manager.go
index ac21c43..f129709 100644
--- c/internal/manager/manager.go
+++ i/internal/manager/manager.go
@@ -250,6 +250,50 @@ func (m *Manager) PrintResult() {
 	fmt.Println(buf.String())
 }

+func (m *Manager) PrintMetrics() {
+	if flags.MetricsFile == "" {
+		return
+	}
+
+	const (
+		metricsFilePermission = 0644
+		metricsDirPermission  = 0755
+	)
+
+	metricsFile, err := fsutils.ExpandDir(flags.MetricsFile)
+	if err != nil {
+		logger.ErrorF("Failed to expand metrics file: %v", err)
+		return
+	}
+	if err = os.MkdirAll(filepath.Dir(metricsFile), metricsDirPermission); err != nil {
+		logger.ErrorF("Failed to create metrics file directory: %v", err)
+		return
+	}
+	f, err := os.OpenFile(metricsFile, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, metricsFilePermission)
+	if err != nil {
+		logger.ErrorF("Failed to open metrics file: %v", err)
+		return
+	}
+	defer f.Close()
+	for labels, count := range m.errors.GetMetrics() {
+		t := strings.Split(labels, ":")
+		if len(t) != 2 {
+			continue
+		}
+		linterID := t[0]
+		ruleID := t[1]
+		fmt.Fprintf(f, `dmt_linter_warnings_count{"version": "%s", "linter": "%s", "rule": "%s"} %d\n`, flags.Version, linterID, ruleID, count)
+	}
+	if err := f.Sync(); err != nil {
+		logger.ErrorF("Failed to sync metrics file: %v", err)
+		return
+	}
+	if err := f.Close(); err != nil {
+		logger.ErrorF("Failed to close metrics file: %v", err)
+		return
+	}
+}
+
 func (m *Manager) HasCriticalErrors() bool {
 	return m.errors.ContainsErrors()
 }
@juev
feat: update README and add metrics printing for linting results
09d957b 
diff --git c/cmd/dmt/main.go i/cmd/dmt/main.go
index b4275de..fa4d581 100644
--- c/cmd/dmt/main.go
+++ i/cmd/dmt/main.go
@@ -43,6 +43,7 @@ func runLint(dir string) {
 	mng := manager.NewManager(dir, cfg)
 	mng.Run()
 	mng.PrintResult()
+	mng.PrintMetrics()

 	metricsClient, err := metrics.NewPrometheusMetricsService(os.Getenv("DMT_METRICS_URL"), os.Getenv("DMT_METRICS_TOKEN"))
 	if err != nil {
diff --git c/internal/manager/manager.go i/internal/manager/manager.go
index f129709..b5b683d 100644
--- c/internal/manager/manager.go
+++ i/internal/manager/manager.go
@@ -282,16 +282,12 @@ func (m *Manager) PrintMetrics() {
 		}
 		linterID := t[0]
 		ruleID := t[1]
-		fmt.Fprintf(f, `dmt_linter_warnings_count{"version": "%s", "linter": "%s", "rule": "%s"} %d\n`, flags.Version, linterID, ruleID, count)
+		fmt.Fprintf(f, "dmt_linter_warnings_count{\"version\": \"%s\", \"linter\": \"%s\", \"rule\": \"%s\"} %d\n", flags.Version, linterID, ruleID, count)
 	}
 	if err := f.Sync(); err != nil {
 		logger.ErrorF("Failed to sync metrics file: %v", err)
 		return
 	}
-	if err := f.Close(); err != nil {
-		logger.ErrorF("Failed to close metrics file: %v", err)
-		return
-	}
 }

 func (m *Manager) HasCriticalErrors() bool {
@juev juev force-pushed the feature/telemetry branch from 2efa6fd to 09d957b Compare March 25, 2025 06:00
@juev juev marked this pull request as ready for review March 25, 2025 10:49
@juev juev requested a review from ldmonster March 25, 2025 10:49
juev added 6 commits March 25, 2025 15:51
@juev
feat: enhance metrics handling for linter warnings and refactor relat…
cd2d273 
…ed functions

Signed-off-by: Evsyukov Denis <[email protected]>
@juev
fix: optimize error handling in GetLinterWarningsCountLabels and clea…
0a673ec 
…n up metrics logging

Signed-off-by: Evsyukov Denis <[email protected]>
@juev
fix: filter out non-warning errors in error processing logic
8a13e8c 
Signed-off-by: Evsyukov Denis <[email protected]>
@juev
feat: move linter warning metrics increment to dedicated processing f…
5451019 
…unction

Signed-off-by: Evsyukov Denis <[email protected]>
@juev
feat: invoke dedicated processing for linter warnings count metrics
3c25629 
Signed-off-by: Evsyukov Denis <[email protected]>
@juev
feat: add linter warnings metrics collection and processing logic
59c8d07 
Signed-off-by: Evsyukov Denis <[email protected]>
ldmonster
ldmonster reviewed Mar 25, 2025
View reviewed changes
ldmonster
ldmonster reviewed Mar 25, 2025
View reviewed changes
ldmonster
ldmonster reviewed Mar 25, 2025
View reviewed changes
juev and others added 2 commits March 25, 2025 18:32
@juev @ldmonster
Update cmd/dmt/main.go
8af65bb 
Co-authored-by: Pavel Okhlopkov <[email protected]>
Signed-off-by: Evsyukov Denis <[email protected]>
@juev
chore: remove unused slices import from main.go
23335b7 
Signed-off-by: Evsyukov Denis <[email protected]>
@ldmonster
Copy link
Contributor

We can parse metric name from Desc() by regex

Output example:

Desc{fqName: "dmt_linter_warnings_count", help: "DMT linter warnings count", constLabels: {}, variableLabels: {version,linter,rule}}

In original prometheus code this field is private, but we still can parse it from description

@juev juev marked this pull request as draft March 26, 2025 06:50
@juev juev marked this pull request as ready for review March 26, 2025 11:39
@juev juev requested a review from ldmonster March 26, 2025 11:39
@juev juev merged commit ba38e47 into main Mar 26, 2025
6 checks passed
@juev juev deleted the feature/telemetry branch March 26, 2025 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ldmonster ldmonster ldmonster approved these changes

@yalosev yalosev Awaiting requested review from yalosev

Assignees

@juev juev

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Telemetry for exceptions
2 participants
@juev @ldmonster