Skip to content

Commit fa83176

Browse files
at15at15
committed
[book] Add chatper prefix to folders
1 parent f7ad20a commit fa83176

File tree

15 files changed

+182
-66
lines changed

15 files changed

+182
-66
lines changed

Diff for: ‎README.md

+2-59
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,11 @@
11
# Notes on Time Series Database
22

3-
This is a work in progress book about time series database (TSDB) that contains:
3+
This is a work in progress [book](book) about time series database (TSDB) that (will) contains:
44

5-
- Survey on [various existing time series databases](https://github.com/xephonhq/awesome-time-series-database)
5+
- [Survey](survey) on [various existing time series databases](https://github.com/xephonhq/awesome-time-series-database)
66
- How to write a (distributed) time series database from scratch.
77
- Related fields like distributed tracing, OLAP database.
88

9-
## Roadmap
10-
11-
Merge survey in other repo
12-
13-
- [ ] [libtsdb-go](https://github.com/libtsdb/libtsdb-go)
14-
- [ ] [xephon-k](https://github.com/xephonhq/xephon-k)
15-
- [ ] [xephon-b](https://github.com/xephonhq/xephon-b)
16-
17-
Layout would be (subject to change, and need to merge w/ layout in libtsdb-go)
18-
19-
- 01 Overview
20-
- time series data
21-
- 02 Basic
22-
- integer, float number and their machine representation (basic for working on compression)
23-
- statistic, distribution, random etc.
24-
- maybe information theory
25-
? data model ? a dedicated chapter?
26-
- 03 components (might flatten this)
27-
- overview
28-
- query language
29-
- complex query execution would also be interesting
30-
- SQL
31-
- continuous query, prometheus ppl also has an article about it
32-
- index (in memory and on disk?)
33-
- compression
34-
- storage
35-
- different backend, pros and cons
36-
- distribution
37-
- 04 survey
38-
- influxdb
39-
- prometheus, victoriametrics, thanos (and another one ... forgot the name from weaveworks)
40-
- m3db
41-
- akumuli, btrdb (both are tree like)
42-
- timescaledb (and others on pg, most of them are dead so ...)
43-
- druid alike (linkedin also got one)
44-
- graphite (and newer backends)
45-
- mysql
46-
- elasticsearch
47-
- crate (it looks pretty good, though it seems it is bought by some other company)
48-
- 05 benchmark
49-
- components
50-
- workload generator
51-
- stats collector
52-
- basic math, distribution
53-
- historgram and lhdr (forgot what's the name ...)
54-
- existing tools, their pro can cons
55-
- 06 application
56-
- iot
57-
- monitoring
58-
- 07 analysis (a.k.a ml a.k.a import ts as tf)
59-
- prediction
60-
- anomaly detection
61-
- 08 tracing (keep it in this repo for now)
62-
- 09 OLAP, MPP
63-
- 10 How we (or maybe just me...) built Xephon-K (not yet)
64-
- failure injection and correctness validation
65-
- continuous benchmark
669

6710
## Related
6811

Diff for: ‎book/01-overview/README.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Chapter 01: Overview
2+
3+
## TODO
4+
5+
- [ ] what is time series data
6+
- [ ] why do we need time series database
7+
- [ ] a glimpse on tsdb landscape (like DDA, I can draw the map by myself and put xephon-k in the far north beyond the wall)

Diff for: ‎book/basic/README.md renamed to ‎book/02-basic/README.md

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
11
# Chapter 02: Basic
22

3-
- [Primitive Data Type](primitive) Representation of primitive type like integer, float, string.
3+
- [Primitive Data Type](primitive) Representation of primitive type like integer, float, string.
4+
5+
## TODO
6+
7+
- [ ] math

Diff for: ‎book/02-basic/primitive/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Primitive Types
2+
3+
- [Endianness](endianness.md)

Diff for: ‎book/basic/primitive/code/endianness.c renamed to ‎book/02-basic/primitive/code/endianness.c

File renamed without changes.

Diff for: ‎book/basic/primitive/endianness.md renamed to ‎book/02-basic/primitive/endianness.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Endianness
22

3+
## TODO
4+
5+
- [ ] what happens when you write a string as file (in c, go), what the layout on disk? btw mention `hexdump`
6+
7+
## Overview
8+
39
- `byte` i.e. 8 bits is the atomic unit, endianness is for byte ordering. (for the scope in this book)
410
- big-endian, save MSB (Most Significant Byte) in the smallest address
511
- little-endian, save LSB (Least Significant Byte) in the smallest address
@@ -8,7 +14,7 @@
814
- numeric literal, left is MSB
915
- bit shift, shift left shift towards MSB, e.g. `0x0400 << 1` is `0x0800`
1016

11-
Example [code](code/endianness.c)
17+
Example [code/endianness.c](code/endianness.c)
1218

1319
`2^10+1` when written in literal is `100_0000_0001` or `0x0401` (hex), it requires two bytes
1420

Diff for: ‎book/README.md

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Notes on Time Series Database, unpublished edition
2+
3+
The layout is subject to change, and need to merge w/ libtsdb-go
4+
5+
- [01 Overview](01-overview)
6+
- what is time series data and why we need time series database
7+
- [02 Basic](02-basic)
8+
- [Endianess](02-basic/endianess.md) how to store larger than one byte value
9+
- integer, float number and their machine representation (basic for working on compression)
10+
- statistic, distribution, random etc.
11+
- maybe information theory
12+
? data model ? a dedicated chapter?
13+
- 03 components (might flatten this)
14+
- overview
15+
- query language
16+
- complex query execution would also be interesting
17+
- SQL
18+
- continuous query, prometheus ppl also has an article about it
19+
- index (in memory and on disk?)
20+
- compression
21+
- storage
22+
- different backend, pros and cons
23+
- distribution
24+
- 04 survey
25+
- influxdb
26+
- prometheus, victoriametrics, thanos (and another one ... forgot the name from weaveworks)
27+
- m3db
28+
- akumuli, btrdb (both are tree like)
29+
- timescaledb (and others on pg, most of them are dead so ...)
30+
- druid alike (linkedin also got one)
31+
- graphite (and newer backends)
32+
- mysql
33+
- elasticsearch
34+
- crate (it looks pretty good, though it seems it is bought by some other company)
35+
- 05 benchmark
36+
- components
37+
- workload generator
38+
- stats collector
39+
- basic math, distribution
40+
- historgram and lhdr (forgot what's the name ...)
41+
- existing tools, their pro can cons
42+
- 06 application
43+
- iot
44+
- monitoring
45+
- 07 analysis (a.k.a ml a.k.a import ts as tf)
46+
- prediction
47+
- anomaly detection
48+
- 08 tracing (keep it in this repo for now)
49+
- 09 OLAP, MPP
50+
- 10 How we (or maybe just me...) built Xephon-K (not yet)
51+
- failure injection and correctness validation
52+
- continuous benchmark

Diff for: ‎book/abbreviation.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@
55
- [ ] support building index for abbreviation and point them to definition
66

77
- MSB: Most Significant Byte
8-
- LSB: Least Significant Byte
8+
- LSB: Least Significant Byte

Diff for: ‎book/basic/primitive/README.md

-1
This file was deleted.

Diff for: ‎book/overview/README.md

-1
This file was deleted.

Diff for: ‎doc/ROADMAP.md

+10-1
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,14 @@
66

77
See [milestones/v0.1.0-primitive](milestones/v0.1.0-primitive). Covers primitive types, math and some common compression algorithm.
88

9-
## Finished
9+
Merge survey in other repo
10+
11+
- [ ] [libtsdb-go](https://github.com/libtsdb/libtsdb-go)
12+
- [ ] [xephon-k](https://github.com/xephonhq/xephon-k)
13+
- [ ] [xephon-b](https://github.com/xephonhq/xephon-b)
14+
15+
### v0.2.0-data model
1016

17+
Covers common data model used in time series databases, and compared it with relational database, document store etc.
18+
19+
## Finished

Diff for: ‎doc/milestones/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Milestones for notes on time series database
2+
3+
- [v0.1.0-primitive](v0.1.0-primitive) primitive types integer, float.

Diff for: ‎doc/milestones/v0.1.0-primitive/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Motivation
44

5-
Have a good understand of layout and int and float. Explain what happens when converting one to another.
5+
Have a good understanding for integer and float. Explain what happens when converting one to another.
66
Might cover gauge, histogram etc. as well.
77

88
## Specs

Diff for: ‎survey/README.md

+9
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,15 @@
22

33
This folder contains survey on implementation of various TSDBs.
44

5+
## TODO
6+
7+
Databases
8+
9+
- [ ] crates (java)
10+
- [ ] questdb (java)
11+
12+
## Format
13+
514
Required list in README
615

716
- link to [awesome tsdb](https://github.com/xephonhq/awesome-time-series-database), which should contain basic meta data.

Diff for: ‎survey/opentracing/README.md

+82
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
# Opentracing
2+
3+
It is not database ... but part of the observability family ...
4+
5+
copied from go.ice ... well it's long time ago
6+
7+
## TODO
8+
9+
- [ ] `opentracing.StartSpanFromContext` is using `opentracing.GlobalTracer()`, what's the better alternative? `SpanFromContext`
10+
- [ ] set global tag for a service, i.e. `aws-region` etc.
11+
- [ ] Is it possible to have cycle in service dependencies, i.e. the cycle is break based on certain flags
12+
13+
## Tutorial
14+
15+
https://github.com/yurishkuro/opentracing-tutorial/tree/master/go
16+
17+
Lesson 01
18+
19+
- `opentracing.GlobalTracer` default is a noop tracer
20+
- tags are meant to describe attributes of the span that apply to the whole duration of the span
21+
- log is for timestamped event
22+
- `LogFields` is for zero allocation (like zap I suppose ...) TODO: I don't think those extra work really boost performance
23+
- `LogKV` takes k,v pair
24+
- [ ] FIXME: typo in README.md, `previosuly` should be `previously`
25+
26+
Lesson 02
27+
28+
- `rootSpan.Tracer().StartSpan("span-name")` would create a entire new trace instead of a child of rootSpan
29+
- `rootSpan.Tracer().StartSpan("span-name", opentracing.ChildOf(rootSpan.Context()))` would create child
30+
- edge in the DAG is `SpanReference`, there are two `ReferenceType`,
31+
- `ChildOf`, rootSpan has a logical dependency on the child before it can complete its operation
32+
- `FollowsFrom` rootSpan is the ancestor, but it does not wait for the child to finish, i.e. a best-effort, fire-and-forget cache write
33+
- [ ] FIXME: typo in README.md line 96, should be `rootSpan.Context()` instead of `span.Context()`
34+
- [x] **FIXED**: fatal `opentracing.StartSpanFromContext` is using `opentracing.GlobalTracer()`
35+
- we should fix this when doing tracing in go.ice, no reason to use `GlobalTracer`
36+
- [ ] there might be fix in following lessons, or existing codebase ...
37+
38+
Lesson 03
39+
40+
- trace both client and server (across process)
41+
- client side need to `Inject` data into carrier (i.e. HTTP Header)
42+
- server side need to `Extract` to start span using the trace information passed in carrier (i.e. HTTP Header)
43+
- [x] TODO: are tags on client side span passed to server? I guess not? no
44+
45+
Lesson 04
46+
47+
- use baggage to pass data along the entire trace
48+
- remember not to have typo in baggage name ....
49+
50+
Finished tutorial is in https://github.com/at15/opentracing-tutorial w/o extra credit
51+
52+
## Example HotRod
53+
54+
- https://github.com/jaegertracing/jaeger/blob/master/examples/hotrod/pkg/tracing/init.go#L31 `Init` creates a new instance of Jaeger tracer
55+
- `SamplerConfig`, source code has detail about sampler types `const`, `probabilistic`, `rateLimiting`, `remote`
56+
- [ ] `SamplingServerURL` ... does this need to be set? seems only needed when using `remote`?
57+
58+
````go
59+
// Init creates a new instance of Jaeger tracer.
60+
func Init(serviceName string, metricsFactory metrics.Factory, logger log.Factory, hostPort string) opentracing.Tracer {
61+
cfg := config.Configuration{
62+
Sampler: &config.SamplerConfig{
63+
Type: "const",
64+
Param: 1,
65+
},
66+
Reporter: &config.ReporterConfig{
67+
LogSpans: false,
68+
BufferFlushInterval: 1 * time.Second,
69+
LocalAgentHostPort: hostPort,
70+
},
71+
}
72+
tracer, _, err := cfg.New(
73+
serviceName,
74+
config.Logger(jaegerLoggerAdapter{logger.Bg()}),
75+
config.Observer(rpcmetrics.NewObserver(metricsFactory, rpcmetrics.DefaultNameNormalizer)),
76+
)
77+
if err != nil {
78+
logger.Bg().Fatal("cannot initialize Jaeger Tracer", zap.Error(err))
79+
}
80+
return tracer
81+
}
82+
````

0 commit comments

Comments
 (0)