Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 40: Serializers Implementation for Schema Registry #41

Merged
merged 100 commits into from
Jul 21, 2020
Merged
Changes from 1 commit
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
76d99a3
copying over changes to new fork
shiveshr Jun 8, 2020
b634867
copying over to new fork
shiveshr Jun 8, 2020
a0876cc
serializers copying over
shiveshr Jun 8, 2020
05cc270
copying over
shiveshr Jun 8, 2020
8ee1fef
contract
shiveshr Jun 8, 2020
193776d
Merge branch 'contract' into serializers
shiveshr Jun 8, 2020
600ff74
remove </p>
shiveshr Jun 8, 2020
be5115e
Merge branch 'master' into contract
shiveshr Jun 9, 2020
409c106
merge with master
shiveshr Jun 9, 2020
88af2e7
remove </p>
shiveshr Jun 10, 2020
b1a8894
Merge branch 'contract' into serializers
shiveshr Jun 10, 2020
1961ba1
add name util and hash util
shiveshr Jun 10, 2020
58e7f65
removing all unwanted auto generated swagger files
shiveshr Jun 11, 2020
026b725
Merge branch 'contract' into serializers
shiveshr Jun 11, 2020
7027d88
rename
shiveshr Jun 11, 2020
4be3f40
PR comment
shiveshr Jun 12, 2020
cc223ec
Merge branch 'contract' into serializers
shiveshr Jun 12, 2020
a0a2db7
marking interfaces as beta
shiveshr Jun 15, 2020
8a8c872
Merge branch 'contract' into serializers
shiveshr Jun 15, 2020
3f64c11
validation rules of list
shiveshr Jun 16, 2020
fba77f6
Merge branch 'contract' into serializers
shiveshr Jun 16, 2020
a449b4b
remove Validation rules of list method
shiveshr Jun 16, 2020
22a0a0d
Merge branch 'contract' into serializers
shiveshr Jun 16, 2020
9a50464
PR comment
shiveshr Jun 16, 2020
8cc3091
PR comment
shiveshr Jun 17, 2020
d063e7c
Merge branch 'contract' into serializers
shiveshr Jun 17, 2020
5a7d639
Removing schema validation rules
shiveshr Jun 18, 2020
a9cca3a
Merge branch 'contract' into serializers
shiveshr Jun 18, 2020
0b6230a
license
shiveshr Jun 18, 2020
911f793
merge with contract
shiveshr Jun 18, 2020
7ff74af
Merge branch 'contract' into serializers
shiveshr Jun 18, 2020
c9f9d3f
license
shiveshr Jun 18, 2020
5d91574
javadoc statement
shiveshr Jun 18, 2020
7c30b4f
Merge branch 'contract' into serializers
shiveshr Jun 18, 2020
98a9c6b
javadoc
shiveshr Jun 18, 2020
dcff47c
Merge branch 'contract' into serializers
shiveshr Jun 18, 2020
28660e4
deserializerAsJson name fix
shiveshr Jun 18, 2020
53b8988
compatibility enum reversal
shiveshr Jun 19, 2020
8e2c704
Merge branch 'contract' into serializers
shiveshr Jun 19, 2020
ce1aa98
addschema
shiveshr Jun 19, 2020
151b808
Merge branch 'contract' into serializers
shiveshr Jun 19, 2020
bffd0f7
javadoc
shiveshr Jun 22, 2020
26e34f1
Merge branch 'contract' into serializers
shiveshr Jun 22, 2020
049fb6d
group id generator
shiveshr Jun 22, 2020
910f870
add error message in model helper
shiveshr Jun 22, 2020
d690a21
Merge branch 'contract' into serializers
shiveshr Jun 22, 2020
a44c0fb
required
shiveshr Jun 22, 2020
a82ee56
Merge branch 'contract' into serializers
shiveshr Jun 22, 2020
7b34571
Merge branch 'serializers' of https://github.com/shiveshr/schema-regi…
shiveshr Jun 22, 2020
48da360
add new deserializer factory method
shiveshr Jun 23, 2020
72919bb
PR comments
shiveshr Jun 24, 2020
48e54e9
Merge branch 'contract' into serializers
shiveshr Jun 24, 2020
7c9ba8e
avro reflect deserializer
shiveshr Jun 25, 2020
214ab54
add test for avro reflect deserializer
shiveshr Jun 25, 2020
2241865
javadoc
shiveshr Jun 25, 2020
456e6bd
Merge branch 'contract' into serializers
shiveshr Jun 25, 2020
9741bd5
Merge branch 'master' into contract
shiveshr Jun 25, 2020
6f7f4d5
Merge branch 'contract' into serializers
shiveshr Jun 25, 2020
87c9ebc
make cache package private
shiveshr Jun 25, 2020
f8fd8ad
javadocs
shiveshr Jun 25, 2020
c98ee7b
remove the hashutil from this branch
shiveshr Jun 25, 2020
c3789ca
minor imp
shiveshr Jun 25, 2020
32933ba
use full name
shiveshr Jun 28, 2020
d8b0675
ofRecord in avro schema
shiveshr Jun 28, 2020
322d3c0
exact type to message match for protobuf generic deserializer
shiveshr Jun 29, 2020
a586f4f
generalize the multiformat deserialization
shiveshr Jun 30, 2020
cdc0c0c
revert rename of json generic object
shiveshr Jun 30, 2020
8693357
refactoring serializer factory into multiple classes for readability
shiveshr Jul 2, 2020
223df77
bug fixes
shiveshr Jul 3, 2020
8c69b3f
PR comments
shiveshr Jul 3, 2020
572f34f
javadoc
shiveshr Jul 5, 2020
cf82e07
encode header as a config param
shiveshr Jul 8, 2020
c4a732b
review comments
shiveshr Jul 8, 2020
5daaeb4
temp
shiveshr Jul 8, 2020
eedee88
merge with issue 46
shiveshr Jul 8, 2020
b63a8a4
merge with master
shiveshr Jul 8, 2020
23e6470
rename variable
shiveshr Jul 8, 2020
4925759
remove validate object
shiveshr Jul 8, 2020
1dafddb
ProtobufSchema
shiveshr Jul 8, 2020
53b8f9c
minor refactoring
shiveshr Jul 10, 2020
a744198
PR comments
shiveshr Jul 10, 2020
c6a02c8
fix
shiveshr Jul 10, 2020
bf0dfb4
PR comments
shiveshr Jul 12, 2020
d70ba7e
merge with master
shiveshr Jul 13, 2020
4117557
PR comment
shiveshr Jul 13, 2020
0f770fa
Merge branch 'master' into serializers
shiveshr Jul 13, 2020
9823974
PR comments
shiveshr Jul 13, 2020
7ee6fe0
PR comments
shiveshr Jul 14, 2020
ff47289
PR comments
shiveshr Jul 14, 2020
6c3fd87
PR comments
shiveshr Jul 15, 2020
2d6711d
adding avro protobuf and json creator methods in WithSchema
shiveshr Jul 15, 2020
ade09af
PR comments, javadoc fix, and return schema string instead of JsonSch…
shiveshr Jul 16, 2020
9cbaaae
checkstyle
shiveshr Jul 16, 2020
c0e57be
PR comment
shiveshr Jul 16, 2020
174c4e6
PR comments
shiveshr Jul 16, 2020
fe0974f
revert rename of registry serializer factory
shiveshr Jul 16, 2020
d464b6b
PR comment - generic T on JSONSchema
shiveshr Jul 17, 2020
9137aa6
Merge with master, use everit json schema
shiveshr Jul 17, 2020
ed740ea
Merge with master
shiveshr Jul 17, 2020
a8fb0dd
PR comments, javadoc corrections, test addition
shiveshr Jul 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
contract
Signed-off-by: Shivesh Ranjan <shivesh.ranjan@gmail.com>
shiveshr committed Jun 8, 2020
commit 8ee1fef5fdc3320577de3ebb4a17db3e0312a03e
Original file line number Diff line number Diff line change
@@ -13,12 +13,10 @@

import javax.annotation.concurrent.GuardedBy;
import java.util.Collection;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Map;
import java.util.NoSuchElementException;
import java.util.Queue;
import java.util.Set;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.function.Function;

@@ -39,34 +37,25 @@ public class ContinuationTokenIterator<T, Token> implements Iterator<T> {
private T next;
@GuardedBy("$lock")
private boolean canHaveNext;
@GuardedBy("$lock")
private final Set<Token> tokens;

public ContinuationTokenIterator(Function<Token, Map.Entry<Token, Collection<T>>> loadingFunction, Token tokenIdentity) {
this.loadingFunction = loadingFunction;
this.queue = new LinkedBlockingQueue<T>();
this.token = tokenIdentity;
this.canHaveNext = true;
this.next = null;
this.tokens = new HashSet<>();
}

@Synchronized
private void load() {
next = next == null ? queue.poll() : next;
while (next == null && canHaveNext) {
Map.Entry<Token, Collection<T>> result = loadingFunction.apply(token);
boolean tokenUpdated = result.getKey() != null && !tokens.contains(result.getKey());
if (result.getKey() != null) {
tokens.add(result.getKey());
}
token = result.getKey();

queue.addAll(result.getValue());
next = queue.poll();
if (next == null) {
canHaveNext = tokenUpdated;
}
canHaveNext = next != null;
}
}

Original file line number Diff line number Diff line change
@@ -28,19 +28,9 @@
public class ContinuationTokenIteratorTest {
@Test
public void test() {
// 1. call method 1st call returns - list of 5 items + new token
// verify that call method is not called until all 10 are read.
// 2. call returns empty list + new token
// 3. call returns empty list + new token
// 4. call returns list of 10 items + new token
// verify that we consume 10 items without calling the callmethod
// 5. call returns empty list + same token. --> this should exit
Queue<ListWithToken> responses = spy(new LinkedBlockingQueue<>());
responses.add(new ListWithToken(Lists.newArrayList(1, 2, 3, 4, 5), "1"));
responses.add(new ListWithToken(Collections.emptyList(), "2"));
responses.add(new ListWithToken(Collections.emptyList(), "3"));
responses.add(new ListWithToken(Lists.newArrayList(6, 7, 8, 9, 10), "4"));
responses.add(new ListWithToken(Collections.emptyList(), "4"));
responses.add(new ListWithToken(Collections.emptyList(), ""));
Function<String, Map.Entry<String, Collection<Integer>>> func = token -> {
ListWithToken result = responses.poll();
return new AbstractMap.SimpleEntry<>(result.token, result.list);
@@ -51,13 +41,7 @@ public void test() {
assertEquals(myIterator.next().intValue(), i + 1);
}
verify(responses, times(1)).poll();
for (int i = 5; i < 10; i++) {
assertTrue(myIterator.hasNext());
assertEquals(myIterator.next().intValue(), i + 1);
}
verify(responses, times(4)).poll();
assertFalse(myIterator.hasNext());
verify(responses, times(5)).poll();
}

@Data
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
/**
* Copyright (c) Dell Inc., or its subsidiaries. All Rights Reserved.
* <p>
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
*
* http://www.apache.org/licenses/LICENSE-2.0
*/
package io.pravega.schemaregistry.contract.v1;