Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.DS_Store
.project
fileDiff/
mutationDiff/
source/
target/
xdcrDiffer
go.mod
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think go.mod and go.sum should be included
Otherwise, I'm getting

------
 > [builder 3/6] RUN make all:
0.169 go build -o xdcrDiffer -v
0.175 go: cannot find main module, but found .git/config in /go
0.175 	to create a module there, run:
0.175 	go mod init
0.176 make: *** [Makefile:17: build] Error 1
------

go.sum
xdcrDiffer.log
45 changes: 45 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
ARG BUILDER_IMAGE=docker-registry-proxy.corp.amdocs.com/golang:1.24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-amdocs folks cannot reach this builder image.

ERROR: failed to solve: docker-registry-proxy.corp.amdocs.com/redhat/ubi9:9.5: failed to resolve source metadata for docker-registry-proxy.corp.amdocs.com/redhat/ubi9:9.5: failed to do request: Head "https://docker-registry-proxy.corp.amdocs.com/v2/redhat/ubi9/manifests/9.5": dialing docker-registry-proxy.corp.amdocs.com:443 container via direct connection because  has no HTTPS proxy: connecting to docker-registry-proxy.corp.amdocs.com:443: dial tcp: lookup docker-registry-proxy.corp.amdocs.com: no such host

Can this be a generic image, like before?
https://github.com/couchbase/xdcrDiffer/pull/116/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R1
ARG BUILDER_IMAGE=docker.io/golang:1.22


ARG FINAL_IMAGE=docker-registry-proxy.corp.amdocs.com/redhat/ubi9:9.5

ARG http_proxy
ARG https_proxy
ARG no_proxy


FROM $BUILDER_IMAGE as builder

ENV http_proxy=${http_proxy}
ENV https_proxy=${https_proxy}
ENV no_proxy=${no_proxy}


ENV LANG=en_US.utf8
ENV LC_ALL=en_US.utf8

ARG TARGETARCH
ARG TARGETOS

ENV GOOS=$TARGETOS
ENV GOARCH=$TARGETARCH
ENV CGO_ENABLED=0

COPY . .

RUN make all

RUN echo "xdcrdiffer:x:10001:10001::/:/xdcrDiffer" > /passwd

RUN go build -ldflags='-s -w -extldflags "-static"' -v \
-o xdcrDiffer main.go

RUN chmod +x ./runDiffer.sh

FROM $FINAL_IMAGE as final

COPY --from=builder /go/xdcrDiffer /
COPY --from=builder /go/runDiffer.sh /
COPY --from=builder /passwd /etc/passwd

USER xdcrdiffer
ENTRYPOINT ["/runDiffer.sh"]
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ If an XDCR is ongoing, it is quite possible that the tool will show documents as
- [Getting Started](#getting-started)
* [Prerequisites](#prerequisites)
* [Compiling](#compiling)
* [Building with Docker](#building-with-docker)
+ [Multiple platform using docker buildx](#multiple-platform-using-docker-buildx)
* [Running](#running)
+ [Preparing Couchbase Clusters](#preparing-couchbase-clusters)
+ [runDiffer](#rundiffer)
Expand Down Expand Up @@ -46,6 +48,33 @@ It can be compiled using the accompanying make file.
~/xdcrDiffer$ make
```

### Building with Docker

Docker build:
```
docker build -t xdcr-differ:1.0.0 .
```

#### Multiple platform using docker buildx

Creating the builder:
(for additional configuration -> https://docs.docker.com/build/buildkit/configure/)

CTX=mycontext
docker buildx ls | grep -w ${CTX} || docker buildx create --name ${CTX}

Docker buildx build for local/developer machine only:
```
docker buildx build --builder mycontext --platform "linux/arm64" --load -t xdcr-differ:1.0.0 .
```

Docker buildx build and image push:
```
docker buildx build --builder mycontext --platform "linux/amd64,linux/arm64" --push -t <myregistry>/xdcr-differ:1.0.0 .
```



### Running
#### Preparing Couchbase Clusters
Before running the differ to examine consistencies between two clusters, it is *highly recommended* to first set the Metadata Purge Interval to a low value, and then once that period has elapsed, run compaction on both clusters to ensure that tombstones are removed. Compaction will also ensure that the differ will only receive the minimum amount of data necessary, which will help minimize the storage requirement for the diff tool.
Expand Down
17 changes: 10 additions & 7 deletions runDiffer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,16 @@ EOF
}

function waitForBgJobs {
local mainPid=$1
local mainPidCnt=$(ps -ef | grep -v grep | grep -c $mainPid)
local jobsCnt=$(jobs -l | grep -c "Running")
while (((($jobsCnt > 0)) && (($mainPidCnt > 0)))); do
sleep 1
jobsCnt=$(jobs -l | grep -c "Running")
mainPidCnt=$(ps -ef | grep -v grep | grep -c $mainPid)
local pid=$1
# Check if the PID is provided
if [[ -z "$pid" ]]; then
echo "Usage: wait_for_pid <pid>"
return 1
fi
# Loop until the process no longer exists
while kill -0 "$pid" 2>/dev/null; do
echo "Waiting for PID $pid to terminate..."
sleep 20
done
}

Expand Down
12 changes: 12 additions & 0 deletions xdcr-differ-pod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v1
kind: Pod
metadata:
name: xdcr-differ-pod
labels:
app: xdcr-differ
spec:
containers:
- name: xdcr-differ
imagePullPolicy: IfNotPresent
image: myRepo/xdcr-differ:1.0.1
command: ["/bin/sleep", "infinity"]