Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DbSearcher overflow #83

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
eae269f
chore: test in jest
zhuangya Apr 14, 2017
546be60
Merge pull request #11 from zhuangya/jest
lionsoul2014 Apr 14, 2017
af9c879
Merge branch 'master' of https://github.com/lionsoul2014/ip2region
lionsoul2014 Apr 19, 2017
e4f5405
Merge pull request #1 from lionsoul2014/master
mohong122 May 25, 2017
979e9bd
Merge pull request #15 from mohong122/master
lionsoul2014 Aug 1, 2017
769ab31
correct post path
dongyado Sep 8, 2017
1ad6274
fix incorrect python test helper.
chengpiaopiao Sep 15, 2017
499bc18
Merge pull request #19 from chengpiaopiao/master
lionsoul2014 Sep 16, 2017
6eec7a2
Update the to data 2017/10/09 version
lionsoul2014 Oct 20, 2017
7f1216e
Merge branch 'master' of http://git.oschina.net/lionsoul/ip2region
lionsoul2014 Oct 20, 2017
0f39924
add python3 support
Oct 20, 2017
4ffb133
!4 python 3正常运行
Oct 21, 2017
b48dbe5
Bug fixed: Adjust the super header length to 8192 for all client impl…
lionsoul2014 Oct 21, 2017
b0aecd7
active the upgrade notice
lionsoul2014 Oct 21, 2017
1f0b07d
Update the notice
lionsoul2014 Oct 21, 2017
a49fae2
Update the README
lionsoul2014 Oct 21, 2017
29bacb0
Update the README
lionsoul2014 Oct 21, 2017
4d0ce55
bold the upgrade notice
lionsoul2014 Oct 21, 2017
5f29715
java binding maven upload & track the latest dbMaker.jar
lionsoul2014 Oct 23, 2017
02391e2
Add the maven repository address
lionsoul2014 Oct 23, 2017
07ea599
Update the REAMD
lionsoul2014 Oct 23, 2017
217a7da
Untrack the binding/java/target/ directory
lionsoul2014 Oct 28, 2017
68540b4
Fix the doc error
lionsoul2014 Nov 23, 2017
39555fc
Update the data to version 2018/01/12
lionsoul2014 Jan 16, 2018
facd3ee
The code computes the average of two integers using either division o…
Jun 20, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,21 @@
*.pyc
*~
META-INF/

# Binary Files #
*.jar
!dbMaker-*.jar

# Package Files #
.settings/
.classpath
.project

# vim swp file #
*.swp

# binding
/binding/java/classes/
/binding/java/doc/
/binding/java/target/
/binding/java/*.jar
71 changes: 43 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
ip2region - 最自由的ip地址查询库,ip到地区的映射库,提供Binary,B树和纯内存三种查询算法,妈妈再也不用担心我的ip地址定位。

**1. 99.9%准确率,定时更新:**
### 1. 99.9%准确率,定时更新:

数据聚合了一些知名ip到地名查询提供商的数据,这些是他们官方的的准确率,经测试着实比纯真啥的准确多了。<br />
每次聚合一下数据需要1-2天,会不定时更新。

**2. 标准化的数据格式:**
### 2. 标准化的数据格式:

每条ip数据段都固定了格式:_城市Id|国家|区域|省份|城市|ISP_

只有中国的数据精确到了城市,其他国家只能定位到国家,后前的选项全部是0,已经包含了全部你能查到的大大小小的国家。
(请忽略前面的城市Id,个人项目需求)

**3. 体积小:**
### 3. 体积小:

生成的数据库文件ip2region.db只有1.5M(1.2版本前是3.5M)

**4. 多查询客户端的支持,0.0x毫秒级别的查询**
### 4. 多查询客户端的支持,0.0x毫秒级别的查询

已经集成的客户端有:java, php, c, python,nodejs,php扩展(支持linux, php5, php7版本已支持),golang。

Expand All @@ -29,43 +29,58 @@ ip2region - 最自由的ip地址查询库,ip到地区的映射库,提供Bina

任何客户端b-tree都比binary算法快,当然Memory算法固然是最快的!

**5. 测试程序:**
java:
maven仓库地址:
```xml
<dependency>
<groupId>org.lionsoul</groupId>
<artifactId>ip2region</artifactId>
<version>1.4</version>
</dependency>
```

cd binding/java
ant all
java -jar ip2region-{version}.jar ./data/ip2region.db
### 5. 测试程序:
java:
```shell
cd binding/java
ant all
java -jar ip2region-{version}.jar ./data/ip2region.db
```

php:

php binding/php/testSearcher ./data/ip2region.db
```shell
php binding/php/testSearcher.php ./data/ip2region.db
```

c:

cd binding/c/
gcc -g -O2 testSearcher.c ip2region.c
./a.out ../../data/ip2region.db
```shell
cd binding/c/
gcc -g -O2 testSearcher.c ip2region.c
./a.out ../../data/ip2region.db
```

python:

python binding/python/testSearcher ./data/ip2region.db
```shell
python binding/python/testSearcher.py ./data/ip2region.db
```

均会看到如下界面:

initializing B-tree ...
+----------------------------------+
| ip2region test script |
| Author: [email protected] |
| Type 'quit' to exit program |
+----------------------------------+
p2region>> 101.105.35.57
2163|中国|华南|广东省|深圳市|鹏博士 in 0.02295 millseconds
```shell
initializing B-tree ...
+----------------------------------+
| ip2region test script |
| Author: [email protected] |
| Type 'quit' to exit program |
+----------------------------------+
p2region>> 101.105.35.57
2163|中国|华南|广东省|深圳市|鹏博士 in 0.02295 millseconds
```

输入ip地址开始测试,第一次会稍微有点慢,在运行命令后面接入binary,memory来尝试其他算法,建议使用b-tree算法,速度和并发需求的可以使用memory算法。

具体集成请参考不同客户端的测试源码。

**6. 如何生成ip2region.db文件**

### 6. 如何生成ip2region.db文件

从ip2region 1.2.2版本开始里面提交了一个dbMaker-{version}.jar的可以执行jar文件,用它来完成这个工作:
* 1, 确保你安装好了java环境(不玩Java的童鞋就自己谷歌找找拉,临时用一用,几分钟的事情)
Expand All @@ -89,4 +104,4 @@ java -jar dbMaker-1.2.2.jar -src ./data/ip.merge.txt -region ./data/global_regio
# 会看到一大片的输出
```

* 5, 数据库文件的结构和原理请阅读 @冬芽 的blog:[“ip2region数据库文件的结构和原理”](http://dongyado.com/tools/2016/08/18/structure-of-ip2region-database-file/)
* 5, 数据库文件的结构和原理请阅读 @冬芽 的blog:[“ip2region数据库文件的结构和原理”](http://dongyado.com/tool/2016/08/18/structure-of-ip2region-database-file/)
4 changes: 3 additions & 1 deletion binding/c/ip2region.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ typedef unsigned int uint_t;
typedef unsigned long ulong_t;

#define INDEX_BLOCK_LENGTH 12
#define TOTAL_HEADER_LENGTH 4096

//thats 8 * 1024
#define TOTAL_HEADER_LENGTH 8192

/*
* ip2region properties struct
Expand Down
2 changes: 1 addition & 1 deletion binding/c_mmap/ip2region.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ typedef unsigned int uint_t;
typedef unsigned long ulong_t;

#define INDEX_BLOCK_LENGTH 12
#define TOTAL_HEADER_LENGTH 4096
#define TOTAL_HEADER_LENGTH 8192

/*
* ip2region properties struct
Expand Down
39 changes: 34 additions & 5 deletions binding/golang/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
golang 实现ip地址查询
###golang 实现ip地址查询

获取
####获取

```
go get github.com/mohong122/ip2region/binding/golang
Expand All @@ -9,7 +9,7 @@ go get github.com/mohong122/ip2region/binding/golang



使用
####使用

```golang

Expand Down Expand Up @@ -38,7 +38,7 @@ func main() {

```

返回对象
####返回对象
```golang
type IpInfo struct {
CityId int64
Expand All @@ -50,11 +50,40 @@ type IpInfo struct {
}
```

性能
#### 性能

|名称|次数|平均耗时|
|---|---|------|
BenchmarkBtreeSearch-4| 200000 | 7715 ns/op
BenchmarkMemorySearch-4| 2000000 | 840 ns/op
BenchmarkBinarySearch-4| 30000 | 42680 ns/op


#### 测试程序

```
cd /binging/golang

go run main.go ../../data/ip2Region.db

Or

go build -o ip2region main.go
./ip2region ../../data/ip2region.db

```

会看到如下界面

```
initializing
+-------------------------------------------------------+
| ip2region test script |
| format 'ip type' |
| type option 'b-tree','binary','memory' default b-tree |
| Type 'quit' to exit program |
+-------------------------------------------------------+
ip2reginon >> 127.0.0.1 memory
0|未分配或者内网IP|0|0|0|0 960.5µs

```
8 changes: 6 additions & 2 deletions binding/golang/ip2Region.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ import (
)

const (
INDEX_BLOCK_LENGTH = 12
TOTAL_HEADER_LENGTH = 4096
INDEX_BLOCK_LENGTH = 12
TOTAL_HEADER_LENGTH = 8192
)

var err error
Expand Down Expand Up @@ -48,6 +48,10 @@ type IpInfo struct {
ISP string
}

func (ip IpInfo)String() string {
return strconv.FormatInt(ip.CityId, 10) + "|" + ip.Country + "|" + ip.Region + "|" + ip.Province + "|" + ip.City + "|" + ip.ISP
}

func getIpInfo(cityId int64, line []byte) IpInfo {

lineSlice := strings.Split(string(line), "|")
Expand Down
70 changes: 70 additions & 0 deletions binding/golang/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package main

import (
"os"
"github.com/mohong122/ip2region/binding/golang"
"bufio"
"fmt"
"strings"
"errors"
"time"

)

func main() {

db := os.Args[1]

_,err:= os.Stat(db)
if os.IsNotExist(err){
panic("not found db " + db)
}

region, err := ip2region.New(db)
defer region.Close()
fmt.Println(`initializing
+-------------------------------------------------------+
| ip2region test script |
| format 'ip type' |
| type option 'b-tree','binary','memory' default b-tree |
| Type 'quit' to exit program |
+-------------------------------------------------------+`)

reader := bufio.NewReader(os.Stdin)
for {
fmt.Print("ip2reginon >> ")
data, _, _ := reader.ReadLine()
begin:= time.Now()
commands := strings.Fields(string(data))
ip := ip2region.IpInfo{}
len := len(commands)
if len == 0{
continue
}

if commands[0] == "quit"{
break
}

if !(len > 1) {
commands = append(commands, "b-tree")
}
switch commands[1] {
case "b-tree":
ip, err = region.BtreeSearch(commands[0])
case "binary":
ip, err = region.BinarySearch(commands[0])
case "memory":
ip, err = region.MemorySearch(commands[0])
default:
err = errors.New("parameter error")
}

if err != nil {

fmt.Println( fmt.Sprintf("\x1b[0;31m%s\x1b[0m",err.Error()))
}else{
fmt.Println( fmt.Sprintf("\x1b[0;32m%s %s\x1b[0m",ip.String(),time.Since(begin).String()))
}
}
}
2 changes: 1 addition & 1 deletion binding/java/build.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
<property name="jars" value="${basedir}"/>
<property name="sources" value="${basedir}/src"/>
<property name="classes" value="${basedir}/classes"/>
<property name="version" value="1.2"/>
<property name="version" value="1.4"/>
<property name="api" value="${basedir}/doc"/>
<mkdir dir="${classes}"/>
<mkdir dir="${api}"/>
Expand Down
Loading