Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IDN (Internationalized domain names) support #626

Open
romap0 opened this issue May 16, 2020 · 5 comments · May be fixed by #640
Open

IDN (Internationalized domain names) support #626

romap0 opened this issue May 16, 2020 · 5 comments · May be fixed by #640
Labels
feature-request help wanted

Comments

@romap0
Copy link

romap0 commented May 16, 2020

Host operating system: output of uname -a

Linux db6787aaa499 4.15.0-34-generic #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 GNU/Linux

blackbox_exporter version: output of blackbox_exporter -version

0.16.0

What is the blackbox.yml module config.

default

What is the prometheus.yml scrape config.

global:
  scrape_interval:     15s
  evaluation_interval: 15s
scrape_configs:
  - job_name: 'blackbox'
    metrics_path: /probe
    params:
      module: [http_2xx]
    static_configs:
      - targets:
        - http://мтр24.рф #doesnt work
        - http://xn--24-7lcqj.xn--p1ai #works
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox-exporter:9115

What logging output did you get from adding &debug=true to the probe URL?

Logs for the probe:
ts=2020-05-16T22:20:32.688701072Z caller=main.go:304 module=http_2xx target=http://мтр24.рф level=info msg="Beginning probe" probe=http timeout_seconds=119.5
ts=2020-05-16T22:20:32.689183006Z caller=http.go:318 module=http_2xx target=http://мтр24.рф level=info msg="Resolving target address" ip_protocol=ip6
ts=2020-05-16T22:20:32.689332368Z caller=http.go:318 module=http_2xx target=http://мтр24.рф level=error msg="Resolution with IP protocol failed" err="lookup мтр24.рф: no such host"
ts=2020-05-16T22:20:32.689371761Z caller=main.go:119 module=http_2xx target=http://мтр24.рф level=error msg="Error resolving address" err="lookup мтр24.рф: no such host"
ts=2020-05-16T22:20:32.689399341Z caller=main.go:304 module=http_2xx target=http://мтр24.рф level=error msg="Probe failed" duration_seconds=0.000408558

What did you do that produced an error?

What did you expect to see?

probe_success 1

What did you see instead?

probe_success 0
@brian-brazil
Copy link
Contributor

Hmm, how does normal Prometheus scraping deal with this? Does Go even support these?

@romap0
Copy link
Author

romap0 commented May 17, 2020

I'm not familiar with Go, but as I tested with net/http module it works.

Code 
package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
)

func main() {
	url := "http://мтр24.рф"
	fmt.Println("URL:>", url)

	resp, err := http.Get(url)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("response status:", resp.StatusCode)

	body, err := ioutil.ReadAll(resp.Body)
	fmt.Println("response body:", string(body))
}
Logs 
URL:> http://мтр24.рф
response status: 200
response body: <!DOCTYPE html><html><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><link rel=icon type=image/png href=/static/favicon.png><title>База МедТехРейс</title><link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Material+Icons" rel=stylesheet><link rel=stylesheet href=/static/site.css><script type=text/javascript>(function(m,e,t,r,i,k,a){m[i]=m[i]||function(){(m[i].a=m[i].a||[]).push(arguments)};
  m[i].l=1*new Date();k=e.createElement(t),a=e.getElementsByTagName(t)[0],k.async=1,k.src=r,a.parentNode.insertBefore(k,a)})
  (window, document, "script", "https://mc.yandex.ru/metrika/tag.js", "ym");

  ym(55376068, "init", {
       clickmap:true,
       trackLinks:true,
       accurateTrackBounce:true
  });</script><noscript><div><img src=https://mc.yandex.ru/watch/55376068 style="position:absolute; left:-9999px;" alt=""></div></noscript><script async src="https://www.googletagmanager.com/gtag/js?id=UA-122643750-11"></script><script>window.dataLayer = window.dataLayer || [];
 function gtag(){dataLayer.push(arguments);}
 gtag('js', new Date());

 gtag('config', 'UA-122643750-11');</script><link href=/static/css/app.1d0b0fa654b68816afdcba08780601a4.css rel=stylesheet></head><body><div id=app></div><script type=text/javascript src=/static/js/manifest.2ae2e69a05c33dfc65f8.js></script><script type=text/javascript src=/static/js/vendor.c2bab8aabbbb38e4d751.js></script><script type=text/javascript src=/static/js/app.5ac42c176cfe32a3fd1f.js></script></body></html>

@brian-brazil
Copy link
Contributor

Okay, so likely our own resolution stuff can't handle this.

@roidelapluie roidelapluie linked a pull request Jun 15, 2020 that will close this issue
Open
@mem
Copy link
Contributor

mem commented Aug 12, 2020

There's a third-party library for Go that deals with IDN.

The way we are dealing with it is by taking the IDN hostnames, applying the punycode transformation and passing that as the target to. For ping and DNS it's just a matter of applying punycode to the input. For TCP we have to split the hostname and the port. For HTTP we parse the URL, punycode the hostname and build a new URL. For DNS we are also transforming the query (so the BBE config has punycoded strings, not UTF8). Transforming regular hostnames using punycode yields back regular hostnames, so you don't have special cases.

The ugly part is that you see punycoded strings where you expect something else.

I could work on a PR to handle this inside BBE if we decide how to handle that case ("it's OK to see punycoded hostnames in logs", "I want to see UTF8 in logs", "...").

@brian-brazil
Copy link
Contributor

I wouldn't worry about the logs, there's an existing PR #640 which is having issues with some of the finer details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request help wanted
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for IDNA roidelapluie/blackbox_exporter
3 participants
@mem @romap0 @brian-brazil