From 964577235818d4811912821e44c3ef8961a226c0 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 12 May 2021 11:34:48 -0700 Subject: [PATCH 01/27] Documentation update Add document about transactional semantics for Alcor Caches in general and spefically about Ignite Caches. --- .../pages/db_services/alcor-transactions.adoc | 162 ++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 docs/modules/ROOT/pages/db_services/alcor-transactions.adoc diff --git a/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc b/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc new file mode 100644 index 000000000..c96a7cf00 --- /dev/null +++ b/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc @@ -0,0 +1,162 @@ +== Transactions in Alcor +Prasad Kommoju + +v.01, 2021-05-05 + +:toc: right + +== Overview + +This document explains the role of transactions in Alcor and provides a set of guidelines for new services and caches to follow so that they conform to the behavior required. + + +== Brief introduction to transactions + +A transaction is a term from database world. Where it refers to some work by the code which changes some persistent data. For the state of the program to be consistent, either all of it is done, or no part of it is done. This property of all or nothing of transactions is called atomicity. + +Another property of transactions is called isolation, which means, even when multiple processes and threads operate on a data item simultaneously, the end result should be as if they were done in a specific order and not in some arbitrarily interleaved order. Furthermore, at any point of time the execution of these changes the data remains in a consistent state. The concepts are the same in Alcor. + +In Alcor there are many data stores to store the state of the system. Currently the data store is Apache Ignite but the general concepts and techniques laid down by these guidelines apply to any data store. In this document, cache and store are used interchangeably where this is no ambiguity. + +Ignite supports three modes of transactions: + + 1) ATMOIC: In DBMS world, this is called AUTO COMMIT. If an individual data store operation succeeds, its effects are made permanent immediately. There is no way to ask the system to undo the changes. If it fails, the attempted changes are thrown away. Code itself will have to handle undoing the changes if some other condition warrants it. + + 2) TRANSACTIONAL: In DBMS world, this is called MANUAL COMMIT. The code will have to start a transaction, and if everything is good, it will have to issue a COMMIT to make the changes permanent. If for any reason, the changes already made need to be undone, then the code issues a ROLLBACK. If the code does not start a transaction, or explicitly ask for a COMMIT, all changes will be thrown away. + + 3) TRANSACTAIONAL_SNAPSHOT: Not applicable for K-V stores, which are what Alcor uses. It is Applicable to SQL objects only. + +== Transactions in Alcor + +The three main requirements of transactions in Alcor cane be summarized as follows: + +1) Single operations +2) Multicache operations +3) Multi microservice operations +4) Reading a snapshot view of cache(s). + +=== Single cache operations +When a cache is modified in some way and some other condition fails because of errors or exceptions, the cache operation will have to reverted to bring the cache to its original state. + +=== Multicache operations +Some operations may require modifying more than one cache at the same time and in atomic manner. If a later cache operation fails, all of the previously successful cache operations will have to be reverted. Ignite requires that all caches under a transaction have the same transactional mode, either all are ATOMIC, or all are TRANSACTIONAL. Trying to mix operations on multiple caches having multiple atomicity modes will result in an exception. + +=== Multi microservice operations +Each micro service has its own data store and works independently of others but in a coordinated manner. When an operation involves atomic modification of caches owned by more than microservice, one failed operation will have to revert previously successful cache operations. + +=== Reading a snapshot view of cache(s) +Even when an operation requires only reading of a cache, in order to avoid certain anomalous conditions a transaction will be required, specifically, a particular form of isolation property of transactions. + +Transactions and their atomicity and isolation properties ensure that the caches are always consistent, but they come at a cost - reduced concurrency and thus performance. + + +This document focuses on (1) and (2). (3) is now handled by handcrafted rollback mechanism, which is not correct or robust, and possibly expensive too. We will address this later by using the mechanisms supported by the underlying data store. + +Also, this document focuses on Ignite because that is what the current data store is. More or less the same approaches will be needed if another data store were to be used. + +== Enabling TRANSACTONAL Atomicity of Caches + +There are two ways add transactional semantics to Alcor code: + +1) Ignite configuration + +2) Basing the cache on a specific kind of base class + + +=== Ignite configuration +In the Ignite configuration file (Kubernetes/services/ignite_config.xml) used in K8S, or local ignite config file (config/ignite-config.xml) in the Alcor source root. + + a) Add a Bean under the property "cacheConfiguration" + + + + + + + + + + +CACHE_NAME should match the name passed to getCache() method of the cache factory. Usually, this is simply getCache(CacheClass.class) but it could be a hard coded one like getCache("dpm_node_info") when the same class is the type of the Value of entry in the cache. + +If CacheClass.class is passed as the name, then value property should be set to the canonical name of that class. For instance, creating a cache with getCache(PortEntity.class) will requires the value to be set to "com.futurewei.alcor.web.entity.port.PortEntity". + +If more than one cache is instantiated as in ICache and getCache called with VType.class, all of them will be known to Ignite by the same name and represent the same physical cache store. If this is not the desired behavior, then different caches of VType instantiations should use a different name in getCache(). + +=== Basing the cache on a specific kind of base class +This is not supported at the time at a finer granularity. The reason is that Ignite does not seem to allow changing the atomicity of a cache after it has been created. + +Regardless of which of the two methods is followed, if the given cache is housed in a class by it self, or along with one or more other caches, the following guidelines apply. In the first case, it is optional but in the second case it is strongly recommended. + +The class definition for the cache will have to + +* Declare a data member of type CacheFactory +* Assign to CacheFactory data member the parameter of type CacheFactoty passed into the constructor, +* Define a public method getTransaction() which returns cacheFactory.getTransaction(). + +This purpose of this guidelines is to minimize code changes outside the class housing the cache object if and when the manner of obtaining a transaction from the underlying data store changes. + +==== Illustration: +---- +public class MyCache { + private ICache myCache; + private CacheFactory cacheFactory; + + @Autowired + public MyCache(CacheFactory cacheFactory) { + this.cacheFactory = cacheFactory; + this.myCache = cacheFactory.getCache(Vtype.class); + } + + ... + + public Transaction getTransaction() { + return cacheFactory.getTransaction(); + } + + ... +} +---- + +== Working under TRANSACATIONAL atomicity + +Alcor code which interfaces to Ignite caches is structured in such a way that all caches opened or created by a specific "connection" to Ignite all share a common "Ignite client". All these caches have a getTransaction() method which ultimately resolves to the same transaction object. + + +Single cache operations should adhere to the following pattern: + +---- + ... + try { + Transaction txn = myCache.getTransaction().start(); + myCache.put(...); + myCache.remove(...); + ... + txn.commit(); + } + catch (...) { + // log a message and anything else required. + // No need to "Undo" the effects of put(), and remove() + } + ... +---- + +Multi cache operations should adhere to the following pattern: + +---- + ... + try { + Transaction txn = firstCache.getTransaction().start(); + firstCache.put(...); + secondCache.remove(...); + ... + txn.commit(); + } + catch (...) { + // log a message and anything else required. + // No need to "Undo" the effects of put(), and remove() + } + ... +---- + + That is, start a transaction on any one of the caches whose changes should be applied in all or nothing manner, do the operations on all of them and commit when conditions are appropriate. If a throw happens, there is no need to "undo" the cache changes. If some other error is detetected, inside the try block, it might be easier to log it and throw an exception so that the data store automatically rollback changes applied by earlier code. From 183c36a75d63562f99cd4b76b696903f41385cc6 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Fri, 21 May 2021 17:44:56 -0700 Subject: [PATCH 02/27] Apply changes suggested in review --- .../pages/db_services/alcor-transactions.adoc | 41 +++++++++---------- 1 file changed, 20 insertions(+), 21 deletions(-) diff --git a/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc b/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc index c96a7cf00..4c6574053 100644 --- a/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc +++ b/docs/modules/ROOT/pages/db_services/alcor-transactions.adoc @@ -1,8 +1,6 @@ -== Transactions in Alcor += Transactions in Alcor Prasad Kommoju - v.01, 2021-05-05 - :toc: right == Overview @@ -41,7 +39,7 @@ When a cache is modified in some way and some other condition fails because of e === Multicache operations Some operations may require modifying more than one cache at the same time and in atomic manner. If a later cache operation fails, all of the previously successful cache operations will have to be reverted. Ignite requires that all caches under a transaction have the same transactional mode, either all are ATOMIC, or all are TRANSACTIONAL. Trying to mix operations on multiple caches having multiple atomicity modes will result in an exception. -=== Multi microservice operations +=== Multi microservice operations Each micro service has its own data store and works independently of others but in a coordinated manner. When an operation involves atomic modification of caches owned by more than microservice, one failed operation will have to revert previously successful cache operations. === Reading a snapshot view of cache(s) @@ -81,10 +79,10 @@ CACHE_NAME should match the name passed to getCache() method of the cache factor If CacheClass.class is passed as the name, then value property should be set to the canonical name of that class. For instance, creating a cache with getCache(PortEntity.class) will requires the value to be set to "com.futurewei.alcor.web.entity.port.PortEntity". -If more than one cache is instantiated as in ICache and getCache called with VType.class, all of them will be known to Ignite by the same name and represent the same physical cache store. If this is not the desired behavior, then different caches of VType instantiations should use a different name in getCache(). +If more than one cache is instantiated as in ICache and getCache called with VType.class, all of them will be known to Ignite by the same name and represent the same physical cache store. If this is not the desired behavior, then different caches of VType instantiations should use a different name in getCache(). === Basing the cache on a specific kind of base class -This is not supported at the time at a finer granularity. The reason is that Ignite does not seem to allow changing the atomicity of a cache after it has been created. +This is not supported at the time at a finer granularity. The reason is that Ignite does not allow changing the atomicity of a cache after it has been created. Regardless of which of the two methods is followed, if the given cache is housed in a class by it self, or along with one or more other caches, the following guidelines apply. In the first case, it is optional but in the second case it is strongly recommended. @@ -99,22 +97,23 @@ This purpose of this guidelines is to minimize code changes outside the class ho ==== Illustration: ---- public class MyCache { - private ICache myCache; - private CacheFactory cacheFactory; - - @Autowired - public MyCache(CacheFactory cacheFactory) { - this.cacheFactory = cacheFactory; - this.myCache = cacheFactory.getCache(Vtype.class); - } - - ... + private ICache myCache; + private CacheFactory cacheFactory; + + @Autowired + public MyCache(CacheFactory cacheFactory) { + this.cacheFactory = cacheFactory; + this.myCache = cacheFactory.getCache(Vtype.class); +} + +... + +public Transaction getTransaction() { + return cacheFactory.getTransaction(); +} + +... - public Transaction getTransaction() { - return cacheFactory.getTransaction(); - } - - ... } ---- From 1b63054e7a24aaf4e9700658a396d213581aa66e Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 21 Jul 2021 09:58:58 -0700 Subject: [PATCH 03/27] Add missing file --- .../performance/Method-list--CPU-ncm.csv | 648 ++++++++++++++++++ 1 file changed, 648 insertions(+) create mode 100644 docs/modules/ROOT/pages/performance/Method-list--CPU-ncm.csv diff --git a/docs/modules/ROOT/pages/performance/Method-list--CPU-ncm.csv b/docs/modules/ROOT/pages/performance/Method-list--CPU-ncm.csv new file mode 100644 index 000000000..f047210e9 --- /dev/null +++ b/docs/modules/ROOT/pages/performance/Method-list--CPU-ncm.csv @@ -0,0 +1,648 @@ +"Method","Time (ms)","Own Time (ms)" +"java.lang.Thread.run() Thread.java","566130","26" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.run() EpollEventLoop.java","334252","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run() FastThreadLocalRunnable.java","334252","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run() SingleThreadEventExecutor.java","334252","0" +"io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run() ThreadExecutorMap.java","334252","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.epollWait() EpollEventLoop.java","332969","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.Native.epollWait(FileDescriptor, EpollEventArray, FileDescriptor, int, int) Native.java","332969","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.Native.epollWait0(int, long, int, int, int, int) Native.java (native)","332969","332969" +"java.lang.Thread.sleep(long) Thread.java (native)","202885","202885" +"org.apache.ignite.internal.client.thin.TcpClientChannel.lambda$initReceiverThread$0() TcpClientChannel.java","197902","10" +"org.apache.ignite.internal.client.thin.TcpClientChannel$$Lambda$632.run()","197902","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel.processNextMessage() TcpClientChannel.java","197892","0" +"java.lang.ref.Reference$ReferenceHandler.run() Reference.java","197880","197880" +"org.apache.catalina.core.StandardServer.await() StandardServer.java","197880","0" +"org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run() TomcatWebServer.java","197880","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel$ByteCountingDataInput.read(byte[], int) TcpClientChannel.java","197168","10" +"java.net.SocketInputStream.read(byte[], int, int) SocketInputStream.java","197157","197157" +"org.apache.ignite.internal.client.thin.TcpClientChannel$ByteCountingDataInput.readInt() TcpClientChannel.java","171678","0" +"java.util.concurrent.ThreadPoolExecutor$Worker.run() ThreadPoolExecutor.java","33928","20" +"io.grpc.internal.ContextRunnable.run() ContextRunnable.java","33907","0" +"io.grpc.internal.SerializingExecutor.run() SerializingExecutor.java","33907","0" +"com.futurewei.alcor.netwconfigmanager.server.grpc.GoalStateProvisionerServer$GoalStateProvisionerImpl$1.onCompleted() GoalStateProvisionerServer.java","32150","0" +"io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed() ServerCallImpl.java","32150","0" +"io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext() ServerImpl.java","32150","0" +"io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onHalfClose() ServerCalls.java","32150","0" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.getVniInLoop(int) GoalStatePersistenceServiceImpl.java","32129","0" +"org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Object, Method, Object[], MethodProxy) CglibAopProxy.java","27585","64" +"org.apache.ignite.internal.client.thin.TcpClientCache.cacheSingleKeyOperation(Object, ClientOperation, Consumer, Function) TcpClientCache.java","27509","10" +"org.apache.ignite.internal.client.thin.ReliableChannel.service(ClientOperation, Consumer, Function) ReliableChannel.java","27486","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel.service(ClientOperation, Consumer, Function) TcpClientChannel.java","27486","0" +"com.futurewei.alcor.netwconfigmanager.cache.VpcResourceCache$$FastClassBySpringCGLIB$$8dfcc42a.invoke(int, Object, Object[]) ","27133","0" +"com.futurewei.alcor.netwconfigmanager.cache.VpcResourceCache$$EnhancerBySpringCGLIB$$829e3684.getResourceMeta(String) ","26855","0" +"com.futurewei.alcor.common.db.ignite.IgniteClientDbCache.get(Object) IgniteClientDbCache.java","26824","0" +"com.futurewei.alcor.netwconfigmanager.cache.VpcResourceCache.getResourceMeta(String) VpcResourceCache.java","26824","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.get(Object) TcpClientCache.java","26824","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel.receive(long, Function) TcpClientChannel.java","26821","0" +"org.apache.ignite.internal.client.thin.ClientUtils.readObject(BinaryInputStream, boolean) ClientUtils.java","26800","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.readObject(PayloadInputChannel) TcpClientCache.java","26800","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.readObject(BinaryInputStream) TcpClientCache.java","26800","0" +"org.apache.ignite.internal.client.thin.TcpClientCache$$Lambda$676.apply(Object)","26800","0" +"org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(BinaryInputStream, ClassLoader, BinaryReaderHandles) GridBinaryMarshaller.java","26800","0" +"org.apache.ignite.internal.client.thin.ClientBinaryMarshaller.deserialize(BinaryInputStream, BinaryReaderHandles) ClientBinaryMarshaller.java","26800","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize() BinaryReaderExImpl.java","26790","546" +"org.apache.ignite.internal.client.thin.ReliableChannel.affinityService(int, Object, ClientOperation, Consumer, Function) ReliableChannel.java","26724","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0() BinaryReaderExImpl.java","26470","2476" +"org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryReaderExImpl) BinaryClassDescriptor.java","25870","149" +"org.apache.ignite.internal.binary.BinaryFieldAccessor.read(Object, BinaryReaderExImpl) BinaryFieldAccessor.java","25870","0" +"org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.read0(Object, BinaryReaderExImpl) BinaryFieldAccessor.java","25870","10" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.readField(int) BinaryReaderExImpl.java","25870","22" +"org.apache.ignite.internal.binary.BinaryUtils.doReadMap(BinaryInputStream, BinaryContext, ClassLoader, BinaryReaderHandlesHolder, boolean, boolean, BinaryMapFactory) BinaryUtils.java","25870","331" +"org.apache.ignite.internal.client.thin.ClientUtils.unwrapBinary(Object, BinaryReaderHandles) ClientUtils.java","25870","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel$ByteCountingDataInput.read(int) TcpClientChannel.java","25814","693" +"org.apache.ignite.internal.binary.BinaryUtils.deserializeOrUnmarshal(BinaryInputStream, BinaryContext, ClassLoader, BinaryReaderHandlesHolder, boolean, boolean) BinaryUtils.java","25310","0" +"org.apache.ignite.internal.binary.BinaryUtils.doReadObject(BinaryInputStream, BinaryContext, ClassLoader, BinaryReaderHandlesHolder) BinaryUtils.java","25310","32" +"org.apache.ignite.internal.binary.BinaryUtils.doReadCollection(BinaryInputStream, BinaryContext, ClassLoader, BinaryReaderHandlesHolder, boolean, boolean, BinaryCollectionFactory) BinaryUtils.java","14337","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.setHandle(Object, int) BinaryReaderExImpl.java","11356","0" +"org.apache.ignite.internal.binary.BinaryReaderHandles.put(int, Object) BinaryReaderHandles.java","11356","2657" +"java.util.HashMap.put(Object, Object) HashMap.java","10515","10515" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryContext, BinaryInputStream, ClassLoader, BinaryReaderHandles, boolean) BinaryReaderExImpl.java","5623","4948" +"java.util.HashSet.add(Object) HashSet.java","1942","1942" +"java.lang.reflect.Field.set(Object, Object) Field.java","1747","1747" +"io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(StreamListener$MessageProducer) ServerCallImpl.java","1734","0" +"io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(StreamListener$MessageProducer) ServerCallImpl.java","1734","0" +"io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext() ServerImpl.java","1734","0" +"java.lang.ClassLoader.loadClass(String) ClassLoader.java","1528","1745" +"com.futurewei.alcor.netwconfigmanager.server.grpc.GoalStateProvisionerServer$GoalStateProvisionerImpl$1.onNext(Object) GoalStateProvisionerServer.java","1475","0" +"com.futurewei.alcor.netwconfigmanager.server.grpc.GoalStateProvisionerServer$GoalStateProvisionerImpl$1.onNext(Goalstate$GoalStateV2) GoalStateProvisionerServer.java","1475","0" +"io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(Object) ServerCalls.java","1475","0" +"java.util.concurrent.ConcurrentHashMap.get(Object) ConcurrentHashMap.java","1169","1169" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.updateGoalState(String, HostGoalState) GoalStatePersistenceServiceImpl.java","1145","0" +"org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(boolean, int, ClassLoader, boolean) BinaryContext.java","1134","0" +"org.apache.ignite.internal.client.thin.TcpIgniteClient$ClientMarshallerContext.getClass(int, ClassLoader) TcpIgniteClient.java","1069","0" +"org.apache.ignite.internal.util.IgniteUtils.forName(String, ClassLoader, IgnitePredicate) IgniteUtils.java","1069","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(long) SingleThreadEventExecutor.java","939","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.setHandle(Object) BinaryReaderExImpl.java","932","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(Runnable) AbstractEventExecutor.java","918","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServer$1.initChannel(Channel) NettyServer.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.register0(ChannelPromise) AbstractChannel.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe$1.run() AbstractChannel.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.callHandlerAdded() AbstractChannelHandlerContext.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelInitializer.handlerAdded(ChannelHandlerContext) ChannelInitializer.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelInitializer.initChannel(ChannelHandlerContext) ChannelInitializer.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(AbstractChannelHandlerContext) DefaultChannelPipeline.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers() DefaultChannelPipeline.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded() DefaultChannelPipeline.java","851","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute() DefaultChannelPipeline.java","851","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport.start(ServerTransportListener) NettyServerTransport.java","730","0" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.populateVpcResourceCache(HostGoalState, Map) GoalStatePersistenceServiceImpl.java","701","0" +"com.futurewei.alcor.common.db.ignite.IgniteClientDbCache.put(Object, Object) IgniteClientDbCache.java","684","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.put(Object, Object) TcpClientCache.java","684","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.(BinaryContext, BinaryInputStream, ClassLoader, BinaryReaderHandles, boolean, boolean) BinaryReaderExImpl.java","674","652" +"org.apache.ignite.internal.client.thin.TcpClientChannel.send(ClientOperation, Consumer) TcpClientChannel.java","665","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport.createHandler(ServerTransportListener, ChannelPromise) NettyServerTransport.java","616","0" +"org.apache.ignite.internal.binary.BinaryUtils.doReadBinaryObject(BinaryInputStream, BinaryContext, boolean) BinaryUtils.java","599","10" +"org.apache.ignite.internal.binary.BinaryUtils.doReadByteArray(BinaryInputStream) BinaryUtils.java","588","0" +"org.apache.ignite.internal.binary.streams.BinaryAbstractInputStream.readByteArray(int) BinaryAbstractInputStream.java","588","588" +"org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.readFixedType(BinaryReaderExImpl) BinaryFieldAccessor.java","568","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.readString(int) BinaryReaderExImpl.java","568","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.readString() BinaryReaderExImpl.java","568","556" +"com.futurewei.alcor.common.logging.Logger.log(Level, String) Logger.java","524","0" +"java.util.logging.Logger.log(Level, String) Logger.java","524","514" +"io.jaegertracing.internal.reporters.RemoteReporter.flush() RemoteReporter.java","425","10" +"io.jaegertracing.internal.reporters.RemoteReporter$1.run() RemoteReporter.java","425","0" +"java.util.TimerThread.run() Timer.java","425","0" +"java.util.concurrent.ArrayBlockingQueue.offer(Object) ArrayBlockingQueue.java","415","415" +"com.futurewei.alcor.netwconfigmanager.cache.ResourceStateCache$$EnhancerBySpringCGLIB$$ee189f20.addResourceState(String, Object) ","365","0" +"com.futurewei.alcor.netwconfigmanager.cache.ResourceStateCache.addResourceState(String, Object) ResourceStateCache.java","354","0" +"com.futurewei.alcor.netwconfigmanager.cache.ResourceStateCache$$FastClassBySpringCGLIB$$d3c76de.invoke(int, Object, Object[]) ","354","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventArray, int) EpollEventLoop.java","344","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.newHandler(ServerTransportListener, ChannelPromise, List, TransportTracer, int, int, int, int, long, long, long, long, long, boolean, long) NettyServerHandler.java","342","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel$ByteCountingDataInput.readLong() TcpClientChannel.java","327","0" +"java.net.SocketOutputStream.write(byte[], int, int) SocketOutputStream.java","324","324" +"org.apache.ignite.internal.client.thin.TcpClientChannel.write(byte[], int) TcpClientChannel.java","324","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.lambda$cacheSingleKeyOperation$26(Object, Consumer, PayloadOutputChannel) TcpClientCache.java","318","0" +"org.apache.ignite.internal.client.thin.TcpClientCache$$Lambda$677.accept(Object)","318","0" +"org.apache.ignite.internal.client.thin.ClientUtils.writeObject(BinaryOutputStream, Object) ClientUtils.java","318","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.writeObject(PayloadOutputChannel, Object) TcpClientCache.java","318","0" +"com.google.protobuf.AbstractMessage.toString() AbstractMessage.java","318","0" +"com.futurewei.alcor.netwconfigmanager.cache.VpcResourceCache.addResourceMeta(VpcResourceMeta) VpcResourceCache.java","308","0" +"com.futurewei.alcor.netwconfigmanager.cache.VpcResourceCache$$EnhancerBySpringCGLIB$$829e3684.addResourceMeta(VpcResourceMeta) ","308","0" +"org.apache.ignite.internal.client.thin.TcpClientCache.lambda$put$0(Object, PayloadOutputChannel) TcpClientCache.java","308","0" +"org.apache.ignite.internal.client.thin.TcpClientCache$$Lambda$678.accept(Object)","308","0" +"org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(Object, boolean) GridBinaryMarshaller.java","306","0" +"org.apache.ignite.internal.client.thin.ClientBinaryMarshaller.marshal(Object) ClientBinaryMarshaller.java","306","0" +"com.google.protobuf.TextFormat.print(MessageOrBuilder, Appendable) TextFormat.java","306","0" +"com.google.protobuf.TextFormat.printToString(MessageOrBuilder) TextFormat.java","306","0" +"com.google.protobuf.TextFormat$Printer.print(MessageOrBuilder, TextFormat$TextGenerator) TextFormat.java","306","10" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(Object, boolean) BinaryWriterExImpl.java","296","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(Object) BinaryWriterExImpl.java","296","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(Object, boolean) BinaryWriterExImpl.java","296","0" +"com.google.protobuf.GeneratedMessageV3.getAllFields() GeneratedMessageV3.java","262","0" +"com.google.protobuf.GeneratedMessageV3.getAllFieldsMutable(boolean) GeneratedMessageV3.java","262","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Goalstate.java","258","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Goalstate.java","258","0" +"com.google.protobuf.AbstractParser.parseFrom(CodedInputStream, ExtensionRegistryLite) AbstractParser.java","258","0" +"com.google.protobuf.AbstractParser.parseFrom(CodedInputStream, ExtensionRegistryLite) AbstractParser.java","258","0" +"io.grpc.MethodDescriptor.parseRequest(InputStream) MethodDescriptor.java","258","0" +"io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(InputStream) ProtoLiteUtils.java","258","0" +"io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(InputStream) ProtoLiteUtils.java","258","0" +"io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parseFrom(CodedInputStream) ProtoLiteUtils.java","258","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2.(CodedInputStream, ExtensionRegistryLite) Goalstate.java","247","0" +"com.google.protobuf.TextFormat$Printer.printField(Descriptors$FieldDescriptor, Object, TextFormat$TextGenerator) TextFormat.java","245","0" +"com.google.protobuf.TextFormat$Printer.printFieldValue(Descriptors$FieldDescriptor, Object, TextFormat$TextGenerator) TextFormat.java","245","0" +"com.google.protobuf.TextFormat$Printer.printSingleField(Descriptors$FieldDescriptor, Object, TextFormat$TextGenerator) TextFormat.java","245","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext, Object) AbstractChannelHandlerContext.java","228","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(Object) DefaultChannelPipeline.java","228","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(Object) AbstractChannelHandlerContext.java","217","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Object) AbstractChannelHandlerContext.java","217","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(ChannelHandlerContext, Object) DefaultChannelPipeline.java","217","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady() AbstractEpollStreamChannel.java","205","0" +"io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ChannelHandlerContext, Object) ByteToMessageDecoder.java","205","0" +"org.apache.ignite.internal.binary.BinaryClassDescriptor.write(Object, BinaryWriterExImpl) BinaryClassDescriptor.java","200","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable.ensureFieldAccessorsInitialized(Class, Class) GeneratedMessageV3.java","191","12" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2$HostResourcesDefaultEntryHolder.() Goalstate.java","190","0" +"org.apache.ignite.internal.binary.BinaryFieldAccessor.write(Object, BinaryWriterExImpl) BinaryFieldAccessor.java","190","22" +"org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.write0(Object, BinaryWriterExImpl) BinaryFieldAccessor.java","190","0" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.processPortStates(HostGoalState) GoalStatePersistenceServiceImpl.java","186","0" +"io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ChannelHandlerContext, ByteBuf, List) ByteToMessageDecoder.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ChannelHandlerContext, ByteBuf, List) ByteToMessageDecoder.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(ChannelHandlerContext, ByteBuf, List) DefaultHttp2ConnectionDecoder.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(ChannelHandlerContext, ByteBuf, Http2FrameListener) DefaultHttp2FrameReader.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(ChannelHandlerContext, ByteBuf, Http2FrameListener) DefaultHttp2FrameReader.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.decode(ChannelHandlerContext, ByteBuf, List) Http2ConnectionHandler.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(ChannelHandlerContext, ByteBuf, List) Http2ConnectionHandler.java","183","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(ChannelHandlerContext, ByteBuf, Http2FrameListener) Http2InboundFrameLogger.java","183","0" +"com.futurewei.alcor.schema.Goalstate.() Goalstate.java","178","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.() Http2ConnectionHandler.java","170","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteMap(Map) BinaryWriterExImpl.java","169","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.writeMapField(Map) BinaryWriterExImpl.java","169","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(Object) BinaryWriterExImpl.java","169","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readHeadersFrame(ChannelHandlerContext, ByteBuf, int, Http2FrameListener) DefaultHttp2FrameReader.java","160","0" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.processNeighborStates(HostGoalState) GoalStatePersistenceServiceImpl.java","155","0" +"java.lang.Class.getMethod(String, Class[]) Class.java","155","155" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$1.processFragment(boolean, ByteBuf, int, Http2FrameListener) DefaultHttp2FrameReader.java","150","0" +"com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(String[], Descriptors$FileDescriptor[], Descriptors$FileDescriptor$InternalDescriptorAssigner) Descriptors.java","145","0" +"com.google.protobuf.GeneratedMessageV3.getMethodOrDie(Class, String, Class[]) GeneratedMessageV3.java","145","0" +"java.lang.reflect.Method.invoke(Object, Object[]) Method.java","140","45" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(ChannelHandlerContext, int, Http2Headers, int, short, boolean, int, boolean) DefaultHttp2ConnectionDecoder.java","139","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onHeadersRead(ChannelHandlerContext, int, Http2Headers, int, short, boolean, int, boolean) Http2InboundFrameLogger.java","139","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.newHandler(ChannelPromise, Http2FrameReader, Http2FrameWriter, ServerTransportListener, List, TransportTracer, int, int, int, int, long, long, long, long, long, boolean, long) NettyServerHandler.java","138","24" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollServerChannel$EpollServerSocketUnsafe.epollInReady() AbstractEpollServerChannel.java","138","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.onHeadersRead(ChannelHandlerContext, int, Http2Headers) NettyServerHandler.java","127","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler$FrameListener.onHeadersRead(ChannelHandlerContext, int, Http2Headers, int, short, boolean, int, boolean) NettyServerHandler.java","127","0" +"io.grpc.netty.shaded.io.grpc.netty.ProtocolNegotiators$1$1PlaintextHandler.handlerAdded(ChannelHandlerContext) ProtocolNegotiators.java","113","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.addLast(EventExecutorGroup, ChannelHandler[]) DefaultChannelPipeline.java","113","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.addLast(ChannelHandler[]) DefaultChannelPipeline.java","113","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.addLast(EventExecutorGroup, String, ChannelHandler) DefaultChannelPipeline.java","113","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.replace(AbstractChannelHandlerContext, String, ChannelHandler) DefaultChannelPipeline.java","113","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.replace(ChannelHandler, String, ChannelHandler) DefaultChannelPipeline.java","113","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.(Descriptors$FieldDescriptor, String, Class, Class, String) GeneratedMessageV3.java","110","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.handlerAdded(ChannelHandlerContext) NettyServerHandler.java","102","0" +"com.futurewei.alcor.schema.Common.() Common.java","101","0" +"com.google.protobuf.AbstractMessageLite.toByteArray() AbstractMessageLite.java","95","0" +"com.google.protobuf.GeneratedMessageLite$SerializedForm.(MessageLite) GeneratedMessageLite.java","95","0" +"com.google.protobuf.GeneratedMessageV3.writeReplace() GeneratedMessageV3.java","95","0" +"org.apache.ignite.internal.binary.BinaryClassDescriptor.writeReplace(Object) BinaryClassDescriptor.java","95","0" +"org.apache.ignite.internal.binary.BinaryMethodWriteReplacer.replace(Object) BinaryMethodWriteReplacer.java","95","0" +"com.google.protobuf.AbstractParser.parseFrom(byte[], ExtensionRegistryLite) AbstractParser.java","90","0" +"com.google.protobuf.AbstractParser.parseFrom(byte[]) AbstractParser.java","90","0" +"com.google.protobuf.AbstractParser.parseFrom(byte[], int, int, ExtensionRegistryLite) AbstractParser.java","90","0" +"com.google.protobuf.AbstractParser.parseFrom(byte[]) AbstractParser.java","90","0" +"com.google.protobuf.AbstractParser.parsePartialFrom(byte[], int, int, ExtensionRegistryLite) AbstractParser.java","90","0" +"com.google.protobuf.DescriptorProtos$FileDescriptorProto.(CodedInputStream, ExtensionRegistryLite, DescriptorProtos$1) DescriptorProtos.java","90","0" +"com.google.protobuf.DescriptorProtos$FileDescriptorProto.(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","90","0" +"com.google.protobuf.DescriptorProtos$FileDescriptorProto.parseFrom(byte[]) DescriptorProtos.java","90","0" +"com.google.protobuf.DescriptorProtos$FileDescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","90","0" +"com.google.protobuf.DescriptorProtos$FileDescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","90","0" +"com.futurewei.alcor.netwconfigmanager.entity.VpcResourceMeta.getResourceMeta(String) VpcResourceMeta.java","89","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.newEpollHandle(RecvByteBufAllocator$ExtendedHandle) AbstractEpollChannel.java","79","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.recvBufAllocHandle() AbstractEpollChannel.java","79","0" +"com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(Parser, ExtensionRegistryLite) CodedInputStream.java","79","0" +"io.grpc.netty.shaded.io.grpc.netty.AbstractNettyHandler.handlerAdded(ChannelHandlerContext) AbstractNettyHandler.java","78","0" +"com.google.protobuf.CodedOutputStream.computeMessageSize(int, MessageLite) CodedOutputStream.java","74","0" +"com.google.protobuf.CodedOutputStream.computeMessageSizeNoTag(MessageLite) CodedOutputStream.java","74","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.directBuffer(int, int) AbstractByteBufAllocator.java","70","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.directBuffer(int) AbstractByteBufAllocator.java","70","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DecoratingHttp2FrameWriter.writeSettings(ChannelHandlerContext, Http2Settings, ChannelPromise) DecoratingHttp2FrameWriter.java","68","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writeSettings(ChannelHandlerContext, Http2Settings, ChannelPromise) DefaultHttp2ConnectionEncoder.java","68","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.handlerAdded(ChannelHandlerContext) Http2ConnectionHandler.java","68","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.(Http2ConnectionHandler, ChannelHandlerContext) Http2ConnectionHandler.java","68","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.sendPreface(ChannelHandlerContext) Http2ConnectionHandler.java","68","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2OutboundFrameLogger.writeSettings(ChannelHandlerContext, Http2Settings, ChannelPromise) Http2OutboundFrameLogger.java","68","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteCollection(Collection) BinaryWriterExImpl.java","67","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.writeCollectionField(Collection) BinaryWriterExImpl.java","67","0" +"java.lang.invoke.MethodHandleNatives.linkCallSite(Object, int, Object, Object, Object, Object, Object[]) MethodHandleNatives.java","65","65" +"io.grpc.netty.shaded.io.grpc.netty.GrpcHttp2HeadersUtils$GrpcHttp2ServerHeadersDecoder.(long) GrpcHttp2HeadersUtils.java","62","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersDecoder.(boolean, long, int) DefaultHttp2HeadersDecoder.java","62","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersDecoder.(boolean, long) DefaultHttp2HeadersDecoder.java","62","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameWriter.() DefaultHttp2FrameWriter.java","61","0" +"com.google.protobuf.GeneratedMessageV3.getField(Descriptors$FieldDescriptor) GeneratedMessageV3.java","58","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerTransport.() NettyServerTransport.java","57","10" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.buffer(int) AbstractByteBufAllocator.java","57","0" +"io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(int, int) PooledByteBufAllocator.java","57","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameWriter.writeSettings(ChannelHandlerContext, Http2Settings, ChannelPromise) DefaultHttp2FrameWriter.java","57","0" +"com.futurewei.alcor.schema.Gateway.() Gateway.java","55","0" +"com.futurewei.alcor.schema.Vpc.() Vpc.java","55","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularEnumFieldAccessor.(Descriptors$FieldDescriptor, String, Class, Class, String) GeneratedMessageV3.java","52","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersEncoder.() DefaultHttp2HeadersEncoder.java","50","0" +"io.grpc.netty.shaded.io.netty.buffer.Unpooled.wrappedBuffer(byte[]) Unpooled.java","50","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2.internalGetFieldAccessorTable() Goalstate.java","48","0" +"java.util.IdentityHashMap.put(Object, Object) IdentityHashMap.java","48","48" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.tryWriteAsHandle(Object) BinaryWriterExImpl.java","48","0" +"org.apache.ignite.internal.binary.BinaryWriterHandles.put(Object, int) BinaryWriterHandles.java","48","0" +"com.google.protobuf.MapEntry.(MapEntry$Metadata, CodedInputStream, ExtensionRegistryLite) MapEntry.java","46","12" +"com.google.protobuf.MapEntry.(MapEntry$Metadata, CodedInputStream, ExtensionRegistryLite, MapEntry$1) MapEntry.java","46","0" +"com.google.protobuf.MapEntry$Metadata$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) MapEntry.java","46","0" +"com.google.protobuf.MapEntry$Metadata$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) MapEntry.java","46","0" +"com.google.protobuf.GeneratedMessageV3.invokeOrDie(Method, Object, Object[]) GeneratedMessageV3.java","45","0" +"io.grpc.internal.StatsTraceContext.newServerContext(List, String, Metadata) StatsTraceContext.java","45","0" +"com.google.protobuf.Descriptors$FileDescriptor.buildFrom(DescriptorProtos$FileDescriptorProto, Descriptors$FileDescriptor[], boolean) Descriptors.java","43","0" +"com.futurewei.alcor.schema.Port$PortConfiguration.getSerializedSize() Port.java","41","0" +"com.futurewei.alcor.schema.Port$PortState.getSerializedSize() Port.java","41","0" +"org.apache.ignite.internal.client.thin.TcpClientChannel$ByteCountingDataInput.readShort() TcpClientChannel.java","41","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena.allocate(PoolThreadCache, int, int) PoolArena.java","35","0" +"com.futurewei.alcor.netwconfigmanager.cache.HostResourceMetadataCache$$EnhancerBySpringCGLIB$$521f3dc0.getTransaction() ","34","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2Headers$PseudoHeaderName.() Http2Headers.java","34","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.ReadOnlyHttp2Headers.serverHeaders(boolean, AsciiString, AsciiString[]) ReadOnlyHttp2Headers.java","34","0" +"java.lang.invoke.LambdaForm$MH.182988318.linkToTargetMethod(Object, int, long, Object) LambdaForm$MH","34","34" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollRecvByteAllocatorHandle.(RecvByteBufAllocator$ExtendedHandle) EpollRecvByteAllocatorHandle.java","34","0" +"io.grpc.internal.AbstractStream$TransportState.requestMessagesFromDeframer(int) AbstractStream.java","33","0" +"io.grpc.internal.MessageDeframer.deliver() MessageDeframer.java","33","0" +"io.grpc.internal.MessageDeframer.request(int) MessageDeframer.java","33","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$Sink$1.run() NettyServerStream.java","33","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.get(GeneratedMessageV3) GeneratedMessageV3.java","33","0" +"io.grpc.internal.CensusStatsModule$ServerTracerFactory.newServerStreamTracer(String, Metadata) CensusStatsModule.java","33","0" +"com.futurewei.alcor.netwconfigmanager.service.impl.GoalStatePersistenceServiceImpl.processVpcStates(HostGoalState) GoalStatePersistenceServiceImpl.java","32","0" +"com.futurewei.alcor.netwconfigmanager.cache.HostResourceMetadataCache$$FastClassBySpringCGLIB$$fc3227be.invoke(int, Object, Object[]) ","32","0" +"java.lang.Integer.valueOf(int) Integer.java","32","32" +"java.lang.String.getBytes(Charset) String.java","32","32" +"io.grpc.netty.shaded.io.grpc.netty.WriteQueue.flush() WriteQueue.java","32","0" +"io.grpc.netty.shaded.io.grpc.netty.WriteQueue$1.run() WriteQueue.java","32","0" +"java.util.concurrent.locks.LockSupport.unpark(Thread) LockSupport.java","30","30" +"org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(Object, Throwable, boolean) GridFutureAdapter.java","30","0" +"org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(Object, Throwable) GridFutureAdapter.java","30","0" +"org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(Object) GridFutureAdapter.java","30","0" +"org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter$Node) GridFutureAdapter.java","30","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersDecoder.(boolean, HpackDecoder) DefaultHttp2HeadersDecoder.java","29","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2CodecUtil.() Http2CodecUtil.java","29","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2HeadersEncoder.() Http2HeadersEncoder.java","26","0" +"jdk.internal.misc.InnocuousThread.run() InnocuousThread.java","26","26" +"com.futurewei.alcor.schema.Port$PortConfiguration.internalGetFieldAccessorTable() Port.java","25","0" +"io.grpc.internal.AbstractServerStream$TransportState.(int, StatsTraceContext, TransportTracer) AbstractServerStream.java","24","0" +"io.grpc.internal.AbstractStream$TransportState.(int, StatsTraceContext, TransportTracer) AbstractStream.java","24","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$TransportState.(NettyServerHandler, EventLoop, Http2Stream, int, StatsTraceContext, TransportTracer, String) NettyServerStream.java","24","0" +"com.futurewei.alcor.schema.Goalstate$ResourceIdType.internalGetFieldAccessorTable() Goalstate.java","24","0" +"com.futurewei.alcor.schema.Subnet$SubnetState.internalGetFieldAccessorTable() Subnet.java","24","0" +"io.grpc.internal.ServerImpl$ServerListenerImpl.transportCreated(ServerTransport) ServerImpl.java","24","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena.allocate(PoolThreadCache, PooledByteBuf, int) PoolArena.java","24","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena.allocateNormal(PooledByteBuf, int, int) PoolArena.java","24","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena$DirectArena.newChunk(int, int, int, int) PoolArena.java","24","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel.(Channel, LinuxSocket, SocketAddress) AbstractEpollStreamChannel.java","24","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollServerSocketChannel.newChildChannel(int, byte[], int, int) EpollServerSocketChannel.java","24","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollSocketChannel.(Channel, LinuxSocket, InetSocketAddress) EpollSocketChannel.java","24","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersEncoder.(Http2HeadersEncoder$SensitivityDetector) DefaultHttp2HeadersEncoder.java","24","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.WeightedFairQueueByteDistributor.(Http2Connection, int) WeightedFairQueueByteDistributor.java","24","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.WeightedFairQueueByteDistributor.(Http2Connection) WeightedFairQueueByteDistributor.java","24","0" +"java.lang.ThreadLocal.get() ThreadLocal.java","24","24" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBuf.() AbstractByteBuf.java","23","10" +"io.grpc.netty.shaded.io.netty.buffer.Unpooled.() Unpooled.java","22","0" +"com.google.protobuf.DescriptorProtos$DescriptorProto.(CodedInputStream, ExtensionRegistryLite, DescriptorProtos$1) DescriptorProtos.java","22","0" +"com.google.protobuf.DescriptorProtos$DescriptorProto.(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","22","0" +"com.google.protobuf.DescriptorProtos$DescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","22","0" +"com.google.protobuf.DescriptorProtos$DescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue() PooledByteBufAllocator.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator$PoolThreadLocalCache.initialValue() PooledByteBufAllocator.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.(PoolArena, PoolArena, int, int, int, int, int) PoolThreadCache.java","22","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal.get() FastThreadLocal.java","22","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocal.initialize(InternalThreadLocalMap) FastThreadLocal.java","22","0" +"com.futurewei.alcor.schema.Neighbor$NeighborState.internalGetFieldAccessorTable() Neighbor.java","22","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularMessageFieldAccessor.(Descriptors$FieldDescriptor, String, Class, Class, String) GeneratedMessageV3.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf.handleRelease(boolean) AbstractReferenceCountedByteBuf.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf.release() AbstractReferenceCountedByteBuf.java","22","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena.free(PoolChunk, ByteBuffer, long, int, PoolThreadCache) PoolArena.java","22","12" +"io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf.deallocate() PooledByteBuf.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl.streamCreated(ServerStream, String, Metadata) ServerImpl.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl.streamCreatedInternal(ServerStream, String, Metadata, Tag) ServerImpl.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl.startCall(ServerStream, String, ServerMethodDefinition, Metadata, Context$CancellableContext, StatsTraceContext, Tag) ServerImpl.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl.startWrappedCall(String, ServerMethodDefinition, ServerStream, Metadata, Context$CancellableContext, Tag) ServerImpl.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl$1StreamCreated.runInContext() ServerImpl.java","22","0" +"io.grpc.internal.ServerImpl$ServerTransportListenerImpl$1StreamCreated.runInternal() ServerImpl.java","22","0" +"com.futurewei.alcor.schema.Vpc$VpcState.getSerializedSize() Vpc.java","22","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteString(String) BinaryWriterExImpl.java","22","0" +"org.apache.ignite.internal.binary.streams.BinaryAbstractOutputStream.writeByteArray(byte[]) BinaryAbstractOutputStream.java","22","10" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.(boolean) DefaultHttp2Connection.java","22","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.(boolean, int) DefaultHttp2Connection.java","22","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.sendResponseHeaders(ChannelHandlerContext, SendResponseHeadersCommand, ChannelPromise) NettyServerHandler.java","22","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.write(ChannelHandlerContext, Object, ChannelPromise) NettyServerHandler.java","22","0" +"io.grpc.netty.shaded.io.grpc.netty.WriteQueue$AbstractQueuedCommand.run(Channel) WriteQueue.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel.write(Object, ChannelPromise) AbstractChannel.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite(Object, ChannelPromise) AbstractChannelHandlerContext.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(Object, ChannelPromise) AbstractChannelHandlerContext.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.write(Object, boolean, ChannelPromise) AbstractChannelHandlerContext.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.write(Object, ChannelPromise) AbstractChannelHandlerContext.java","22","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.write(Object, ChannelPromise) DefaultChannelPipeline.java","22","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DecoratingHttp2FrameWriter.writeHeaders(ChannelHandlerContext, int, Http2Headers, int, boolean, ChannelPromise) DecoratingHttp2FrameWriter.java","22","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writeHeaders(ChannelHandlerContext, int, Http2Headers, int, boolean, ChannelPromise) DefaultHttp2ConnectionEncoder.java","22","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.writeHeaders(ChannelHandlerContext, int, Http2Headers, int, short, boolean, int, boolean, ChannelPromise) DefaultHttp2ConnectionEncoder.java","22","0" +"com.google.protobuf.Descriptors$FileDescriptor.(DescriptorProtos$FileDescriptorProto, Descriptors$FileDescriptor[], Descriptors$DescriptorPool, boolean) Descriptors.java","22","12" +"org.apache.ignite.internal.binary.BinaryFieldAccessor$DefaultFinalClassAccessor.mode(Object) BinaryFieldAccessor.java","22","22" +"org.apache.ignite.internal.binary.streams.BinaryHeapOutputStream.(int, BinaryMemoryAllocatorChunk) BinaryHeapOutputStream.java","22","22" +"java.lang.reflect.Field.get(Object) Field.java","22","22" +"org.apache.ignite.internal.binary.BinaryUtils.doReadString(BinaryInputStream) BinaryUtils.java","22","12" +"com.google.protobuf.MapEntryLite.parseEntry(CodedInputStream, MapEntryLite$Metadata, ExtensionRegistryLite) MapEntryLite.java","22","0" +"com.google.protobuf.MapEntryLite.parseField(CodedInputStream, ExtensionRegistryLite, WireFormat$FieldType, Object) MapEntryLite.java","22","0" +"com.yourkit.probes.builtin.Databases$Connection_or_Statement_close_Probe.onEnter(Object) a","21","21" +"java.util.LinkedHashMap.get(Object) LinkedHashMap.java","21","21" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.flush() AbstractChannel.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AbstractUnsafe.flush0() AbstractChannel.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.flush() AbstractChannelHandlerContext.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeFlush() AbstractChannelHandlerContext.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeFlush0() AbstractChannelHandlerContext.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelOutboundBuffer.remove() ChannelOutboundBuffer.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelOutboundBuffer.removeBytes(long) ChannelOutboundBuffer.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.flush(ChannelHandlerContext) DefaultChannelPipeline.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.flush0() AbstractEpollChannel.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel.doWrite(ChannelOutboundBuffer) AbstractEpollStreamChannel.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel.doWriteMultiple(ChannelOutboundBuffer) AbstractEpollStreamChannel.java","21","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel.writeBytesMultiple(ChannelOutboundBuffer, IovArray) AbstractEpollStreamChannel.java","21","0" +"io.grpc.stub.ServerCalls$ServerCallStreamObserverImpl.onCompleted() ServerCalls.java","20","10" +"com.futurewei.alcor.netwconfigmanager.cache.HostResourceMetadataCache.addResourceMeta(ResourceMeta) HostResourceMetadataCache.java","20","0" +"com.futurewei.alcor.netwconfigmanager.cache.HostResourceMetadataCache$$EnhancerBySpringCGLIB$$521f3dc0.addResourceMeta(ResourceMeta) ","20","0" +"com.google.protobuf.ByteString.copyFromUtf8(String) ByteString.java","20","0" +"org.apache.ignite.internal.util.IgniteUtils.newHashSet(int) IgniteUtils.java","20","10" +"com.futurewei.alcor.schema.Neighbor$NeighborConfiguration.getSerializedSize() Neighbor.java","20","10" +"com.futurewei.alcor.schema.Neighbor$NeighborState.getSerializedSize() Neighbor.java","20","0" +"org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run() TaskThread.java","20","0" +"io.jaegertracing.internal.reporters.RemoteReporter$QueueProcessor.run() RemoteReporter.java","20","0" +"java.util.concurrent.ConcurrentHashMap.remove(Object) ConcurrentHashMap.java","20","20" +"com.futurewei.alcor.schema.Port$PortState.internalGetFieldAccessorTable() Port.java","16","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBuf.writeBytes(byte[], int, int) AbstractByteBuf.java","16","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBuf.writeBytes(byte[]) AbstractByteBuf.java","16","0" +"io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(int, byte[], int, int) UnpooledUnsafeDirectByteBuf.java","16","0" +"io.grpc.netty.shaded.io.netty.buffer.Unpooled.directBuffer(int) Unpooled.java","13","0" +"io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(int, int) UnpooledByteBufAllocator.java","13","13" +"io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory.() ResourceLeakDetectorFactory.java","12","0" +"io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory$DefaultResourceLeakDetectorFactory.() ResourceLeakDetectorFactory.java","12","0" +"io.grpc.netty.shaded.io.netty.util.ResourceLeakDetectorFactory$DefaultResourceLeakDetectorFactory$1.(ResourceLeakDetectorFactory$DefaultResourceLeakDetectorFactory) ResourceLeakDetectorFactory.java","12","12" +"com.futurewei.alcor.common.db.ignite.IgniteClientTransaction.start() IgniteClientTransaction.java","12","0" +"com.futurewei.alcor.common.utils.CommonUtil.() CommonUtil.java","12","0" +"com.futurewei.alcor.netwconfigmanager.cache.HostResourceMetadataCache.getTransaction() HostResourceMetadataCache.java","12","0" +"com.futurewei.alcor.netwconfigmanager.entity.ResourceMeta.addPortId(String) ResourceMeta.java","12","0" +"com.futurewei.alcor.netwconfigmanager.util.NetworkConfigManagerUtil.splitClusterToHostGoalState(Goalstate$GoalStateV2) NetworkConfigManagerUtil.java","12","0" +"com.futurewei.alcor.netwconfigmanager.util.NetworkConfigManagerUtil$1.() NetworkConfigManagerUtil.java","12","0" +"com.futurewei.alcor.schema.Common$ResourceType.() Common.java","12","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2.internalGetMapField(int) Goalstate.java","12","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2.internalGetSecurityGroupStates() Goalstate.java","12","0" +"com.futurewei.alcor.schema.Goalstate$GoalStateV2$SecurityGroupStatesDefaultEntryHolder.() Goalstate.java","12","0" +"com.futurewei.alcor.schema.Goalstate$HostResources.toBuilder() Goalstate.java","12","0" +"com.futurewei.alcor.schema.Goalstate$HostResources.toBuilder() Goalstate.java","12","0" +"com.futurewei.alcor.schema.Neighbor$NeighborConfiguration.internalGetFieldAccessorTable() Neighbor.java","12","0" +"com.futurewei.alcor.schema.SecurityGroup$SecurityGroupState.() SecurityGroup.java","12","0" +"com.futurewei.alcor.schema.Vpc$VpcConfiguration.getSerializedSize() Vpc.java","12","0" +"com.futurewei.alcor.schema.Vpc$VpcState.internalGetFieldAccessorTable() Vpc.java","12","0" +"com.google.common.base.Stopwatch.() Stopwatch.java","12","0" +"com.google.common.base.Stopwatch.createUnstarted() Stopwatch.java","12","0" +"com.google.common.collect.Lists.computeArrayListCapacity(int) Lists.java","12","0" +"com.google.common.collect.Lists.newArrayList(Object[]) Lists.java","12","0" +"com.google.protobuf.DescriptorProtos$FileOptions.() DescriptorProtos.java","12","0" +"com.google.protobuf.DescriptorProtos$FileOptions.() DescriptorProtos.java","12","0" +"com.google.protobuf.GeneratedMessageV3$ExtendableMessage.() GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$MapFieldAccessor.(Descriptors$FieldDescriptor, String, Class, Class) GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$MapFieldAccessor.get(GeneratedMessageV3) GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$MapFieldAccessor.getMapField(GeneratedMessageV3) GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$MapFieldAccessor.getRepeatedCount(GeneratedMessageV3) GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$RepeatedFieldAccessor.get(GeneratedMessageV3) GeneratedMessageV3.java","12","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$RepeatedMessageFieldAccessor.(Descriptors$FieldDescriptor, String, Class, Class) GeneratedMessageV3.java","12","0" +"com.google.protobuf.MapEntry.newBuilderForType() MapEntry.java","12","0" +"com.google.protobuf.MapField.convertKeyAndValueToMessage(Object, Object) MapField.java","12","0" +"com.google.protobuf.MapField.convertMapToList(MapField$MutatabilityAwareMap) MapField.java","12","0" +"com.google.protobuf.MapField.getList() MapField.java","12","0" +"com.google.protobuf.MapField$ImmutableMessageConverter.convertKeyAndValueToMessage(Object, Object) MapField.java","12","0" +"com.google.protobuf.TextFormat.() TextFormat.java","12","0" +"com.google.protobuf.TextFormat$TextGenerator.eol() TextFormat.java","12","12" +"com.google.protobuf.UnknownFieldSet.getSerializedSize() UnknownFieldSet.java","12","12" +"io.grpc.internal.AbstractServerStream.(WritableBufferAllocator, StatsTraceContext) AbstractServerStream.java","12","0" +"io.grpc.internal.AbstractStream$TransportState.messagesAvailable(StreamListener$MessageProducer) AbstractStream.java","12","0" +"io.grpc.internal.CensusStatsModule$ServerTracer.(CensusStatsModule, TagContext) CensusStatsModule.java","12","0" +"io.grpc.internal.CensusTracingModule$ServerTracerFactory.newServerStreamTracer(String, Metadata) CensusTracingModule.java","12","0" +"io.grpc.internal.KeepAliveManager.(KeepAliveManager$KeepAlivePinger, ScheduledExecutorService, long, long, boolean) KeepAliveManager.java","12","0" +"io.grpc.internal.MessageDeframer.processBody() MessageDeframer.java","12","0" +"io.grpc.internal.ServerCallImpl.newServerStreamListener(ServerCall$Listener) ServerCallImpl.java","12","0" +"io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.(ServerCallImpl, ServerCall$Listener, Context$CancellableContext) ServerCallImpl.java","12","0" +"io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener.messagesAvailable(StreamListener$MessageProducer) ServerImpl.java","12","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler.(ChannelPromise, Http2Connection, ServerTransportListener, List, TransportTracer, Http2ConnectionDecoder, Http2ConnectionEncoder, Http2Settings, int, long, long, long, long, long, KeepAliveEnforcer) NettyServerHandler.java","12","12" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerStream.(Channel, NettyServerStream$TransportState, Attributes, String, StatsTraceContext, TransportTracer) NettyServerStream.java","12","0" +"io.grpc.netty.shaded.io.netty.bootstrap.ServerBootstrap$ServerBootstrapAcceptor.channelRead(ChannelHandlerContext, Object) ServerBootstrap.java","12","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.heapBuffer(int, int) AbstractByteBufAllocator.java","12","0" +"io.grpc.netty.shaded.io.netty.buffer.AbstractByteBufAllocator.heapBuffer() AbstractByteBufAllocator.java","12","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolChunk.(PoolArena, Object, int, int, int, int, int) PoolChunk.java","12","12" +"io.grpc.netty.shaded.io.netty.buffer.Unpooled.buffer() Unpooled.java","12","0" +"io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(int, int) UnpooledByteBufAllocator.java","12","12" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel.(Channel) AbstractChannel.java","12","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel.(Channel, LinuxSocket, SocketAddress) AbstractEpollChannel.java","12","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollSocketChannel.newUnsafe() EpollSocketChannel.java","12","0" +"io.grpc.netty.shaded.io.netty.channel.epoll.EpollSocketChannel.newUnsafe() EpollSocketChannel.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder$1.cumulate(ByteBufAllocator, ByteBuf, ByteBuf) ByteToMessageDecoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.DefaultHeaders.(HashingStrategy, ValueConverter) DefaultHeaders.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.DefaultHeaders$NameValidator.() DefaultHeaders.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http.HttpResponseStatus.() HttpResponseStatus.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http.HttpResponseStatus.(int, String, boolean) HttpResponseStatus.java","12","12" +"io.grpc.netty.shaded.io.netty.handler.codec.http.HttpResponseStatus.newStatus(int, String) HttpResponseStatus.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.CharSequenceMap.() CharSequenceMap.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.CharSequenceMap.(boolean) CharSequenceMap.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.CharSequenceMap.(boolean, ValueConverter) CharSequenceMap.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DecoratingHttp2ConnectionEncoder.remoteSettings(Http2Settings) DecoratingHttp2ConnectionEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$ConnectionStream.(DefaultHttp2Connection) DefaultHttp2Connection.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.addStream(DefaultHttp2Connection$DefaultStream) DefaultHttp2Connection.java","12","12" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.createStream(int, boolean) DefaultHttp2Connection.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultEndpoint.createStream(int, boolean) DefaultHttp2Connection.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.(DefaultHttp2Connection, int, Http2Stream$State) DefaultHttp2Connection.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onSettingsRead(ChannelHandlerContext, Http2Settings) DefaultHttp2ConnectionDecoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$PrefaceFrameListener.onSettingsRead(ChannelHandlerContext, Http2Settings) DefaultHttp2ConnectionDecoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.remoteSettings(Http2Settings) DefaultHttp2ConnectionEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readSettingsFrame(ChannelHandlerContext, ByteBuf, Http2FrameListener) DefaultHttp2FrameReader.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersEncoder.(Http2HeadersEncoder$SensitivityDetector, HpackEncoder) DefaultHttp2HeadersEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController.initialWindowSize(int) DefaultHttp2RemoteFlowController.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$WritabilityMonitor.initialWindowSize(int) DefaultHttp2RemoteFlowController.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.(long, int) HpackDecoder.java","12","12" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.(long) HpackDecoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackEncoder.(boolean, int, int) HpackEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackEncoder.(boolean) HpackEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackEncoder.() HpackEncoder.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.closeStream(Http2Stream, ChannelFuture) Http2ConnectionHandler.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.closeStreamLocal(Http2Stream, ChannelFuture) Http2ConnectionHandler.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler$PrefaceDecoder.decode(ChannelHandlerContext, ByteBuf, List) Http2ConnectionHandler.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onSettingsRead(ChannelHandlerContext, Http2Settings) Http2InboundFrameLogger.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.WeightedFairQueueByteDistributor$State.(WeightedFairQueueByteDistributor, int, Http2Stream, int) WeightedFairQueueByteDistributor.java","12","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.WeightedFairQueueByteDistributor$State.(WeightedFairQueueByteDistributor, Http2Stream, int) WeightedFairQueueByteDistributor.java","12","0" +"io.opencensus.stats.NoopStats.newNoopMeasureMap() NoopStats.java","12","0" +"io.opencensus.stats.NoopStats$NoopStatsRecorder.newMeasureMap() NoopStats.java","12","0" +"io.opencensus.trace.SpanContext.() SpanContext.java","12","0" +"java.lang.ClassLoader.checkPackageAccess(Class, ProtectionDomain) ClassLoader.java","12","12" +"java.lang.invoke.LambdaForm$MH.979943848.linkToTargetMethod(Object, long, Object) LambdaForm$MH","12","12" +"java.util.Collections$UnmodifiableCollection$1.next() Collections.java","12","12" +"org.apache.ignite.internal.binary.BinaryClassDescriptor.preWrite(BinaryWriterExImpl, Object) BinaryClassDescriptor.java","12","0" +"org.apache.ignite.internal.binary.BinaryReaderExImpl.streamPosition(int) BinaryReaderExImpl.java","12","12" +"org.apache.ignite.internal.binary.streams.BinaryHeapOutputStream.(int) BinaryHeapOutputStream.java","12","0" +"org.apache.ignite.internal.binary.streams.BinaryHeapOutputStream.ensureCapacity(int) BinaryHeapOutputStream.java","12","0" +"org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk.reallocate(byte[], int) BinaryMemoryAllocatorChunk.java","12","12" +"org.apache.ignite.internal.client.thin.PayloadOutputChannel.(ClientChannel) PayloadOutputChannel.java","12","0" +"org.apache.ignite.internal.client.thin.TcpClientTransactions.tx() TcpClientTransactions.java","12","0" +"org.apache.ignite.internal.client.thin.TcpClientTransactions.txStart(TransactionConcurrency, TransactionIsolation) TcpClientTransactions.java","12","0" +"org.apache.ignite.internal.client.thin.TcpClientTransactions.txStart0(TransactionConcurrency, TransactionIsolation, Long, String) TcpClientTransactions.java","12","12" +"io.grpc.netty.shaded.io.netty.channel.unix.Socket.accept(byte[]) Socket.java","11","0" +"io.grpc.internal.CompositeReadableBuffer.addBuffer(ReadableBuffer) CompositeReadableBuffer.java","11","0" +"io.grpc.internal.CompositeReadableBuffer.close() CompositeReadableBuffer.java","11","0" +"io.grpc.internal.MessageDeframer.readRequiredBytes() MessageDeframer.java","11","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolArena$DirectArena.newByteBuf(int) PoolArena.java","11","0" +"io.grpc.internal.MessageDeframer.(MessageDeframer$Listener, Decompressor, int, StatsTraceContext, TransportTracer) MessageDeframer.java","11","0" +"io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.() ByteToMessageDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.() HpackDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.CodecOutputList.() CodecOutputList.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.touch(Object, AbstractChannelHandlerContext) DefaultChannelPipeline.java","10","0" +"io.grpc.internal.AbstractReadableBuffer.readInt() AbstractReadableBuffer.java","10","0" +"io.grpc.internal.CompositeReadableBuffer.advanceBufferIfNecessary() CompositeReadableBuffer.java","10","0" +"io.grpc.internal.CompositeReadableBuffer.execute(CompositeReadableBuffer$ReadOperation, int) CompositeReadableBuffer.java","10","0" +"io.grpc.internal.CompositeReadableBuffer.readUnsignedByte() CompositeReadableBuffer.java","10","0" +"io.grpc.internal.MessageDeframer.processHeader() MessageDeframer.java","10","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyReadableBuffer.close() NettyReadableBuffer.java","10","0" +"com.google.protobuf.DescriptorProtos$FieldDescriptorProto.() DescriptorProtos.java","10","0" +"org.apache.ignite.internal.binary.streams.BinaryAbstractInputStream.readByte() BinaryAbstractInputStream.java","10","10" +"io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.createSubPageCaches(int, int, PoolArena$SizeClass) PoolThreadCache.java","10","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache$MemoryRegionCache.() PoolThreadCache.java","10","0" +"org.apache.ignite.internal.binary.streams.BinaryAbstractOutputStream.unsafeEnsure(int) BinaryAbstractOutputStream.java","10","10" +"io.grpc.netty.shaded.io.grpc.netty.AbstractNettyHandler.sendInitialConnectionWindow() AbstractNettyHandler.java","10","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.add(PoolArena, PoolChunk, ByteBuffer, long, int, PoolArena$SizeClass) PoolThreadCache.java","10","0" +"io.grpc.netty.shaded.io.netty.buffer.PoolThreadCache.cache(PoolArena, int, PoolArena$SizeClass) PoolThreadCache.java","10","0" +"io.grpc.netty.shaded.io.netty.util.ReferenceCountUtil.release(Object) ReferenceCountUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.util.ReferenceCountUtil.safeRelease(Object) ReferenceCountUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$HeadersBlockBuilder.headers() DefaultHttp2FrameReader.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2HeadersDecoder.decodeHeaders(int, ByteBuf) DefaultHttp2HeadersDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.decode(int, ByteBuf, Http2Headers, boolean) HpackDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.decode(ByteBuf, HpackDecoder$Sink) HpackDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.HpackDecoder.readName(int) HpackDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$1.(DefaultHttp2FrameReader, int, ChannelHandlerContext, int, short, boolean, int, Http2Flags) DefaultHttp2FrameReader.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$HeadersContinuation.(DefaultHttp2FrameReader, DefaultHttp2FrameReader$1) DefaultHttp2FrameReader.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader$HeadersContinuation.(DefaultHttp2FrameReader) DefaultHttp2FrameReader.java","10","0" +"io.grpc.internal.AbstractServerStream.close(Status, Metadata) AbstractServerStream.java","10","0" +"io.grpc.internal.ServerCallImpl.close(Status, Metadata) ServerCallImpl.java","10","0" +"io.grpc.internal.ServerCallImpl.closeInternal(Status, Metadata) ServerCallImpl.java","10","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$Sink.writeTrailers(Metadata, boolean, Status) NettyServerStream.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.writeStringField(String) BinaryWriterExImpl.java","10","10" +"io.opencensus.tags.TagValue.create(String) TagValue.java","10","0" +"io.opencensus.tags.TagValue.isValid(String) TagValue.java","10","0" +"io.grpc.internal.DeprecatedCensusConstants.() DeprecatedCensusConstants.java","10","0" +"io.opencensus.contrib.grpc.metrics.RpcMeasureConstants.() RpcMeasureConstants.java","10","0" +"com.futurewei.alcor.netwconfigmanager.server.grpc.IpInterceptor.interceptCall(ServerCall, Metadata, ServerCallHandler) IpInterceptor.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.(DefaultChannelPipeline, EventExecutor, String, Class) AbstractChannelHandlerContext.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask.isSkippable(Class, String, Class[]) ChannelHandlerMask.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask.mask(Class) ChannelHandlerMask.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask.mask0(Class) ChannelHandlerMask.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask$2.run() ChannelHandlerMask.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelHandlerMask$2.run() ChannelHandlerMask.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelHandlerContext.(DefaultChannelPipeline, EventExecutor, String, ChannelHandler) DefaultChannelHandlerContext.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.newContext(EventExecutorGroup, String, ChannelHandler) DefaultChannelPipeline.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection.verifyKey(Http2Connection$PropertyKey) DefaultHttp2Connection.java","10","10" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.getProperty(Http2Connection$PropertyKey) DefaultHttp2Connection.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.unconsumedBytes(Http2Stream) DefaultHttp2ConnectionDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead(ChannelHandlerContext, int, ByteBuf, int, boolean) DefaultHttp2ConnectionDecoder.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2FrameReader.readDataFrame(ChannelHandlerContext, ByteBuf, int, Http2FrameListener) DefaultHttp2FrameReader.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2LocalFlowController.state(Http2Stream) DefaultHttp2LocalFlowController.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2LocalFlowController.unconsumedBytes(Http2Stream) DefaultHttp2LocalFlowController.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onDataRead(ChannelHandlerContext, int, ByteBuf, int, boolean) Http2InboundFrameLogger.java","10","0" +"io.grpc.ServerInterceptors$InterceptCallHandler.startCall(ServerCall, Metadata) ServerInterceptors.java","10","0" +"java.security.AccessController.doPrivileged(PrivilegedExceptionAction) AccessController.java (native)","10","0" +"org.apache.ignite.internal.binary.BinaryClassDescriptor.postWrite(BinaryWriterExImpl) BinaryClassDescriptor.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.postWrite(boolean, boolean) BinaryWriterExImpl.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterSchemaHolder.write(BinaryOutputStream, int, boolean) BinaryWriterSchemaHolder.java","10","10" +"java.util.HashMap$EntryIterator.next() HashMap.java","10","10" +"io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil.hexDump(byte[], int, int) ByteBufUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil$HexUtil.() ByteBufUtil.java","10","10" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel.toString() AbstractChannel.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelId.asShortText() DefaultChannelId.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2FrameLogger.logSettings(Http2FrameLogger$Direction, ChannelHandlerContext, Http2Settings) Http2FrameLogger.java","10","0" +"io.grpc.netty.shaded.io.netty.util.internal.logging.AbstractInternalLogger.log(InternalLogLevel, String, Object[]) AbstractInternalLogger.java","10","0" +"io.grpc.netty.shaded.io.netty.util.internal.logging.LocationAwareSlf4JLogger.debug(String, Object[]) LocationAwareSlf4JLogger.java","10","0" +"org.slf4j.helpers.MessageFormatter.arrayFormat(String, Object[]) MessageFormatter.java","10","0" +"com.google.protobuf.CodedInputStream$ArrayDecoder.readRawVarint32() CodedInputStream.java","10","10" +"com.google.protobuf.DescriptorProtos$EnumDescriptorProto.(CodedInputStream, ExtensionRegistryLite, DescriptorProtos$1) DescriptorProtos.java","10","0" +"com.google.protobuf.DescriptorProtos$EnumDescriptorProto.(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","10","0" +"com.google.protobuf.DescriptorProtos$EnumDescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","10","0" +"com.google.protobuf.DescriptorProtos$EnumDescriptorProto$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) DescriptorProtos.java","10","0" +"org.apache.ignite.internal.binary.BinaryContext.updateMetadata(int, BinaryMetadata, boolean) BinaryContext.java","10","0" +"org.apache.ignite.internal.client.thin.TcpIgniteClient$ClientBinaryMetadataHandler.addMeta(int, BinaryType, boolean) TcpIgniteClient.java","10","0" +"com.futurewei.alcor.schema.Subnet$SubnetConfiguration.internalGetFieldAccessorTable() Subnet.java","10","0" +"com.google.protobuf.CodedInputStream$ArrayDecoder.readBytes() CodedInputStream.java","10","0" +"com.futurewei.alcor.schema.Port$PortConfiguration.getDeviceIdBytes() Port.java","10","0" +"com.google.protobuf.GeneratedMessageV3.hasField(Descriptors$FieldDescriptor) GeneratedMessageV3.java","10","0" +"com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$SingularFieldAccessor.has(GeneratedMessageV3) GeneratedMessageV3.java","10","0" +"com.futurewei.alcor.common.utils.CommonUtil.isNullOrEmpty(String) CommonUtil.java","10","0" +"java.lang.String.trim() String.java","10","10" +"java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$1.hasNext() Collections.java","10","10" +"com.futurewei.alcor.schema.Subnet.() Subnet.java","10","0" +"com.google.protobuf.Descriptors$Descriptor.getFields() Descriptors.java","10","0" +"com.google.protobuf.MapEntry.getAllFields() MapEntry.java","10","0" +"java.util.Collections.unmodifiableList(List) Collections.java","10","10" +"com.google.protobuf.Descriptors$Descriptor.crossLink() Descriptors.java","10","0" +"com.google.protobuf.Descriptors$FieldDescriptor.crossLink() Descriptors.java","10","0" +"com.google.protobuf.Descriptors$FileDescriptor.crossLink() Descriptors.java","10","0" +"io.micrometer.core.instrument.binder.logging.MetricsTurboFilter.decide(Marker, Logger, Level, String, Object[], Throwable) LogbackMetrics.java","10","0" +"io.micrometer.core.instrument.Counter.increment() Counter.java","10","0" +"io.micrometer.core.instrument.cumulative.CumulativeCounter.increment(double) CumulativeCounter.java","10","10" +"com.futurewei.alcor.schema.DHCP.() DHCP.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.DefaultHttp2ConnectionEncoder.validateHeadersSentState(Http2Stream, Http2Headers, boolean, boolean) DefaultHttp2ConnectionEncoder.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.handles() BinaryWriterExImpl.java","10","10" +"java.util.HashSet.(int) HashSet.java","10","10" +"com.futurewei.alcor.schema.Neighbor$NeighborConfiguration$FixedIp.getSerializedSize() Neighbor.java","10","0" +"com.futurewei.alcor.schema.Neighbor$NeighborType.() Neighbor.java","10","0" +"com.google.protobuf.Descriptors$Descriptor.(DescriptorProtos$DescriptorProto, Descriptors$FileDescriptor, Descriptors$Descriptor, int) Descriptors.java","10","0" +"com.google.protobuf.Descriptors$Descriptor.(DescriptorProtos$DescriptorProto, Descriptors$FileDescriptor, Descriptors$Descriptor, int, Descriptors$1) Descriptors.java","10","0" +"com.google.protobuf.Descriptors$FieldDescriptor.() Descriptors.java","10","0" +"com.google.protobuf.WireFormat$FieldType.() WireFormat.java","10","0" +"io.grpc.internal.AbstractServerStream$TransportState.closeListener(Status) AbstractServerStream.java","10","0" +"io.grpc.internal.AbstractServerStream$TransportState.complete() AbstractServerStream.java","10","0" +"io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener.closed(Status) ServerImpl.java","10","0" +"io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener.closedInternal(Status) ServerImpl.java","10","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler$6.operationComplete(Future) NettyServerHandler.java","10","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerHandler$6.operationComplete(ChannelFuture) NettyServerHandler.java","10","0" +"io.grpc.netty.shaded.io.grpc.netty.NettyServerStream$TransportState.complete() NettyServerStream.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.AbstractChannel.flush() AbstractChannel.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelPromise) ChannelOutboundBuffer.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline.flush() DefaultChannelPipeline.java","10","0" +"io.grpc.netty.shaded.io.netty.channel.DefaultChannelPromise.trySuccess() DefaultChannelPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.tryPromise() Http2CodecUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.trySuccess(Void) Http2CodecUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2CodecUtil$SimpleChannelPromiseAggregator.trySuccess(Object) Http2CodecUtil.java","10","0" +"io.grpc.netty.shaded.io.netty.handler.codec.http2.Http2ConnectionHandler.flush(ChannelHandlerContext) Http2ConnectionHandler.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.notifyListener0(Future, GenericFutureListener) DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.notifyListeners() DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.notifyListenersNow() DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.setSuccess0(Object) DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.setValue0(Object) DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.DefaultPromise.trySuccess(Object) DefaultPromise.java","10","0" +"io.grpc.netty.shaded.io.netty.util.internal.PromiseNotificationUtil.trySuccess(Promise, Object, InternalLogger) PromiseNotificationUtil.java","10","0" +"io.jaegertracing.internal.reporters.RemoteReporter$FlushCommand.execute() RemoteReporter.java","10","0" +"com.futurewei.alcor.schema.Goalstate$HostResources.internalGetFieldAccessorTable() Goalstate.java","10","0" +"com.futurewei.alcor.schema.Port$PortConfiguration.getMacAddressBytes() Port.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.(BinaryContext) BinaryWriterExImpl.java","10","0" +"org.apache.ignite.internal.binary.BinaryWriterExImpl.(BinaryContext, BinaryThreadLocalContext) BinaryWriterExImpl.java","10","0" +"com.google.protobuf.CodedOutputStream.computeTagSize(int) CodedOutputStream.java","10","10" +"com.google.protobuf.CodedOutputStream.computeUInt32Size(int, int) CodedOutputStream.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.AbstractScheduledEventExecutor.pollScheduledTask(long) AbstractScheduledEventExecutor.java","10","10" +"io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.fetchFromScheduledTaskQueue() SingleThreadEventExecutor.java","10","0" +"java.lang.String.(byte[], int, int, Charset) String.java","10","10" +"java.util.concurrent.ConcurrentHashMap.put(Object, Object) ConcurrentHashMap.java","10","10" +"com.futurewei.alcor.schema.Port$PortConfiguration.(CodedInputStream, ExtensionRegistryLite) Port.java","10","10" +"com.futurewei.alcor.schema.Port$PortConfiguration$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortConfiguration$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState.(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState$1.parsePartialFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState$Builder.mergeFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState$Builder.mergeFrom(CodedInputStream, ExtensionRegistryLite) Port.java","10","0" +"com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(MessageLite$Builder, ExtensionRegistryLite) CodedInputStream.java","10","0" +"com.google.protobuf.MapField$MutatabilityAwareMap.get(Object) MapField.java","10","0" +"java.util.Collections$UnmodifiableMap.get(Object) Collections.java","10","0" +"com.google.protobuf.ByteString.isEmpty() ByteString.java","10","10" +"com.futurewei.alcor.schema.Port$PortConfiguration.writeTo(CodedOutputStream) Port.java","10","0" +"com.futurewei.alcor.schema.Port$PortState.writeTo(CodedOutputStream) Port.java","10","0" +"com.google.protobuf.CodedOutputStream$ArrayEncoder.writeMessage(int, MessageLite) CodedOutputStream.java","10","0" +"com.google.protobuf.CodedOutputStream$ArrayEncoder.writeMessageNoTag(MessageLite) CodedOutputStream.java","10","0" +"java.util.Collections$UnmodifiableList.get(int) Collections.java","10","10" +"io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.pollTask() SingleThreadEventExecutor.java","10","0" +"io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.pollTaskFrom(Queue) SingleThreadEventExecutor.java","10","10" +"java.util.concurrent.ArrayBlockingQueue.take() ArrayBlockingQueue.java","10","10" +"java.lang.ThreadLocal.set(Object) ThreadLocal.java","10","10" +"org.apache.ignite.internal.util.IgniteUtils.restoreOldIgniteName(String, String) IgniteUtils.java","10","0" From 93e468ce5ad790a9c4d63b98fc3a9fea4a1457ce Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Fri, 22 Oct 2021 10:36:12 -0700 Subject: [PATCH 04/27] SQLFeildQuery changes (work in progress) --- lib/pom.xml | 6 + .../common/db/ignite/IgniteClientDbCache.java | 204 +++++++++++++++++- .../alcor/web/entity/route/Router.java | 4 +- 3 files changed, 207 insertions(+), 7 deletions(-) diff --git a/lib/pom.xml b/lib/pom.xml index 6ea9d0e1d..5467210c5 100644 --- a/lib/pom.xml +++ b/lib/pom.xml @@ -143,6 +143,12 @@ Copyright(c) 2020 Futurewei Cloud 2.1.1 compile + + com.google.guava + guava + 29.0-jre + compile + diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index ac0b95508..4558cbf4f 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -22,25 +22,33 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.db.ignite.query.ScanQueryBuilder; import com.futurewei.alcor.common.logging.Logger; import com.futurewei.alcor.common.logging.LoggerFactory; +import com.google.common.reflect.TypeToken; import org.apache.ignite.binary.BinaryObject; import org.apache.ignite.cache.CachePeekMode; +import org.apache.ignite.cache.QueryEntity; +import org.apache.ignite.cache.QueryIndex; import org.apache.ignite.cache.query.Query; import org.apache.ignite.cache.query.QueryCursor; import org.apache.ignite.cache.query.ScanQuery; +import org.apache.ignite.cache.query.SqlFieldsQuery; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import org.apache.ignite.client.ClientCache; import org.apache.ignite.client.ClientCacheConfiguration; import org.apache.ignite.client.ClientException; import org.apache.ignite.client.IgniteClient; import org.apache.ignite.configuration.CacheConfiguration; +import org.apache.ignite.internal.processors.query.QueryUtils; import org.apache.ignite.lang.IgniteBiPredicate; +import org.checkerframework.checker.units.qual.C; import org.springframework.util.Assert; import javax.cache.Cache; import javax.cache.expiry.ExpiryPolicy; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Set; +import javax.swing.*; +import javax.xml.validation.TypeInfoProvider; +import java.lang.reflect.Field; +import java.lang.reflect.Type; +import java.util.*; import java.util.logging.Level; import java.util.stream.Collectors; @@ -53,7 +61,13 @@ public class IgniteClientDbCache implements IgniteICache { public IgniteClientDbCache(IgniteClient igniteClient, String name) { try { - this.cache = igniteClient.getOrCreateCache(name); + Map sqlFields = getSqlFields(); + if (!sqlFields.isEmpty()) { + this.cache = getOrCreateIndexedCache(igniteClient, sqlFields, name); + } + if (this.cache == null) + this.cache = igniteClient.getOrCreateCache(name); + logger.log(Level.INFO, "Cache " + name + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } catch (ClientException e) { logger.log(Level.WARNING, "Create cache for client " + name + " failed:" + e.getMessage()); @@ -69,7 +83,13 @@ public IgniteClientDbCache(IgniteClient igniteClient, CacheConfiguration cacheCo clientCacheConfig.setName(cacheConfig.getName()); clientCacheConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); logger.log(Level.INFO, "Getting or creating cache " + clientCacheConfig.getName() + " AtomicityMode is " + clientCacheConfig.getAtomicityMode()); - this.cache = igniteClient.getOrCreateCache(clientCacheConfig); + Map sqlFields = getSqlFields(); + if (!sqlFields.isEmpty()) { + this.cache = getOrCreateIndexedCache(igniteClient, sqlFields, clientCacheConfig); + } + if (this.cache == null) + this.cache = igniteClient.getOrCreateCache(clientCacheConfig); + logger.log(Level.INFO, "Retrieved cache " + this.cache.getConfiguration().getName() + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } catch (ClientException e) { logger.log(Level.WARNING, "Create cache for client " + cacheConfig.getName() + " failed:" + e.getMessage()); @@ -91,6 +111,88 @@ public IgniteClientDbCache(IgniteClient igniteClient, String name, ExpiryPolicy this.transaction = new IgniteClientTransaction(igniteClient); } + /** + * + * @param igniteClient + * @param name + * If the class has QuerySqlField annotations, add query entry fields and indexes. + */ + private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Map sqlFields, String name) { + logger.log(Level.INFO, "Creating cache " + name + " with index"); + ClientCacheConfiguration ccConfig = new ClientCacheConfiguration(); + ccConfig.setName(name); + return getOrCreateIndexedCacheInternal(igniteClient, sqlFields, ccConfig); + } + + private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Map sqlFields, ClientCacheConfiguration cacheConfig) { + logger.log(Level.INFO, "Creating cache " + cacheConfig.getName() + " with index"); + ClientCacheConfiguration ccConfig = new ClientCacheConfiguration(); + ccConfig.setName(cacheConfig.getName()); + ccConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); + return getOrCreateIndexedCacheInternal(igniteClient, sqlFields, ccConfig); + } + + + private Map getSqlFields() { + Map sqlFields = new HashMap<>(); + Type t = getClass(); + logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + t.getTypeName()); + Class type = (Class) (new TypeToken(getClass()){}.getType()); + logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + type.getName()); + // go through all fields but pick the very first one, for now, + // make it work for multiple fileds later. + for (Field f : type.getDeclaredFields()) { + QuerySqlField annot = f.getAnnotation(QuerySqlField.class); + if (annot.notNull()) { + logger.log(Level.INFO, "annotation: " + annot.toString()); + if (annot.index()) { + if (sqlFields.keySet().isEmpty()) + sqlFields.put("index", annot.name()); + } + else if (sqlFields.values().isEmpty()) + sqlFields.put("value", annot.name()); + } + } + logger.log(Level.INFO, "Found " + sqlFields.size() + " sqlFields"); + return sqlFields; + } + + private ClientCache getOrCreateIndexedCacheInternal(IgniteClient igniteClient, Map sqlFields, ClientCacheConfiguration cachConfig) { + + QueryEntity qryEnt = new QueryEntity(); + String idxFld = sqlFields.get("index"); + String valFld = sqlFields.get("value"); + + Class type = (Class) (new TypeToken(getClass()){}.getType()); + String className = type.getName(); + logger.log(Level.INFO, "IndexedCache: " + cachConfig.getName() + " IDX " + idxFld + " VAL " + valFld + " TYP " + className); + try { + qryEnt.setKeyType(String.valueOf(type.getField(idxFld))); + qryEnt.setValueType(type.getClass().getName()); + qryEnt.addQueryField(idxFld, String.valueOf(type.getField(idxFld)), null); + qryEnt.addQueryField(valFld, String.valueOf(type.getField(valFld)), null); + QueryIndex qryIndex = new QueryIndex(idxFld); + qryEnt.setIndexes(Collections.singleton(qryIndex)); + } + catch (Exception e) { + logger.log(Level.WARNING, "Failed to create index on cache: " + cachConfig.getName() + ": " + e.getMessage()); + return null; + } + + cachConfig.setQueryEntities(qryEnt); + + // also make not of this cache, somewhere, somehow? + // have a static cache? + ClientCache cache = igniteClient.getOrCreateCache(cachConfig); + + String result = cache == null ? "FAILED" : "WORKED"; + + logger.log(Level.INFO, "Creating index on " + cachConfig.getName() + " " + result); + + return cache; + } + + @Override public V get(K key) throws CacheException { try { @@ -161,6 +263,10 @@ public boolean remove(K key) throws CacheException { @Override public V get(Map filterParams) throws CacheException { + Map sqlFields = checkForSqlFieldsQuery(filterParams); + if (sqlFields != null) + return getSqlFields(sqlFields, filterParams); + IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); return get(predicate); } @@ -189,6 +295,9 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE @Override public Map getAll(Map filterParams) throws CacheException { + Map sqlFields = checkForSqlFieldsQuery(filterParams); + if (sqlFields != null) + return getSqlFieldsAll(sqlFields, filterParams); IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); return getAll(predicate); } @@ -223,4 +332,87 @@ public long size() { public Transaction getTransaction() { return transaction; } + + private Map checkForSqlFieldsQuery(Map filterParams) { + Map sqlFields = new HashMap<>(); + Class type = (Class) (new TypeToken(getClass()) {}.getType()); + for (Field f : type.getDeclaredFields()) { + QuerySqlField annot = f.getAnnotation(QuerySqlField.class); + if (annot.notNull()) { + if (annot.index()) + sqlFields.put("key", annot.name()); + else + sqlFields.put("value", annot.name()); + } + } + + /* + * There must be exactly two sqlfileds, one for the index lookup + * and the other for the _VAL (V) field in the Class declaration. + * There can only one be exactly one field in queryParams for now. + * The entry in the filterparams should be an indexed field. + * If these conditions are true, run SQLFieldsQuery otherwise, ScanQuery. + */ + if (sqlFields.size() == 2 && filterParams.size() == 1 && sqlFields.containsKey(filterParams.keySet())) { + return sqlFields; + } + + return null; + } + + private V getSqlFields(Map sqlFields, Map filterParams) { + try { + Map result = runSQLFieldsQuery(sqlFields, filterParams); + if (result == null || result.isEmpty()) + return null; + return (V) result.get(0); + } + catch (Exception e) { + return null; + } + } + + public Map getSqlFieldsAll(Map sqlFields, Map filterParams) { + try { + Map values = runSQLFieldsQuery(sqlFields, filterParams); + return values; + } + catch (Exception e) { + return null; + } + } + + /* + * SELECT value_sqlfield + * FROM "ClassNameOfTheCache".classnameofthecache + * WHERE key_sqlfield = filterParam.value + */ + private Map runSQLFieldsQuery(Map sqlFields, Map filterParams) throws CacheException { + + String sql = buildSqlFieldsQuery(sqlFields, filterParams); + SqlFieldsQuery query = new SqlFieldsQuery(sql); + Map results = new HashMap<>(); + + try (QueryCursor> cursor = cache.query(query)) { + for (List row : cursor) { + results.put((K)row.get(0), (V)row.get(1)); + } + return results; + } + catch (ClientException e) { + logger.log(Level.WARNING, "SqlFieldsQuery error: " + e.getMessage()); + throw new CacheException(e.getMessage()); + } + } + + private String buildSqlFieldsQuery(Map sqlFields, Map filterParams){ + StringBuilder sb = new StringBuilder("select "); + sb.append(sqlFields.get("value")).append(" from \""); + sb.append(cache.getConfiguration().getName()).append("\""); + sb.append(" where "); + sb.append(filterParams.keySet()).append(" = "); + sb.append(filterParams.values()); + + return sb.toString(); + } } \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java index 3fb644037..80dbe2041 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java @@ -18,6 +18,7 @@ free of charge, to any person obtaining a copy of this software and associated d import com.fasterxml.jackson.annotation.JsonProperty; import com.futurewei.alcor.common.entity.CustomerResource; import lombok.Data; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import org.springframework.data.annotation.CreatedDate; import org.springframework.data.annotation.LastModifiedDate; @@ -43,6 +44,7 @@ public class Router extends CustomerResource { private String vpcDefaultRouteTableId; // store vpc_id + @QuerySqlField @JsonProperty("owner") private String owner; @@ -106,4 +108,4 @@ public Router(Router r) { r.getTenantId(), true, r.getStatus(), r.getRouterExtraAttributeId(), r.getVpcDefaultRouteTableId()); } -} +} \ No newline at end of file From 0b1395f61934fa2c6222d12f25fde9c308232392 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 27 Oct 2021 19:40:41 -0700 Subject: [PATCH 05/27] Create one index works --- .../db/ignite/IgniteClientCacheFactory.java | 10 +- .../common/db/ignite/IgniteClientDbCache.java | 307 ++++++++++-------- .../alcor/common/utils/CommonUtil.java | 24 +- .../alcor/web/entity/route/Router.java | 2 +- 4 files changed, 204 insertions(+), 139 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientCacheFactory.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientCacheFactory.java index 799def6a4..d41c09ef5 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientCacheFactory.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientCacheFactory.java @@ -47,23 +47,23 @@ public IgniteClientCacheFactory(IgniteClient igniteClient, int interval, int exp @Override public ICache getCache(Class v) { - return new IgniteClientDbCache<>(igniteClient, v.getName()); + return new IgniteClientDbCache(igniteClient, v, v.getName()); } @Override public ICache getCache(Class v, String cacheName) { - return new IgniteClientDbCache<>(igniteClient, cacheName); + return new IgniteClientDbCache(igniteClient, v, cacheName); } @Override public ICache getCache(Class v, CacheConfiguration cacheConfig) { - return new IgniteClientDbCache<>(igniteClient, cacheConfig); + return new IgniteClientDbCache<>(igniteClient, v, cacheConfig); } @Override public ICache getExpireCache(Class v, long timeout, TimeUnit timeUnit) { ExpiryPolicy ep = CreatedExpiryPolicy.factoryOf(new Duration(timeUnit, timeout)).create(); - return new IgniteClientDbCache<>(igniteClient, v.getName(), ep); + return new IgniteClientDbCache<>(igniteClient, v, v.getName(), ep); } @Override @@ -75,4 +75,4 @@ public IDistributedLock getDistributedLock(Class t) { public Transaction getTransaction() { return new IgniteClientTransaction(igniteClient); } -} +} \ No newline at end of file diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 4558cbf4f..2338ac801 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -22,7 +22,10 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.db.ignite.query.ScanQueryBuilder; import com.futurewei.alcor.common.logging.Logger; import com.futurewei.alcor.common.logging.LoggerFactory; +import com.futurewei.alcor.common.utils.CommonUtil; +import com.futurewei.alcor.common.utils.ControllerUtil; import com.google.common.reflect.TypeToken; +import io.netty.util.concurrent.EventExecutorChooserFactory; import org.apache.ignite.binary.BinaryObject; import org.apache.ignite.cache.CachePeekMode; import org.apache.ignite.cache.QueryEntity; @@ -37,18 +40,14 @@ free of charge, to any person obtaining a copy of this software and associated d import org.apache.ignite.client.ClientException; import org.apache.ignite.client.IgniteClient; import org.apache.ignite.configuration.CacheConfiguration; -import org.apache.ignite.internal.processors.query.QueryUtils; import org.apache.ignite.lang.IgniteBiPredicate; -import org.checkerframework.checker.units.qual.C; import org.springframework.util.Assert; import javax.cache.Cache; import javax.cache.expiry.ExpiryPolicy; -import javax.swing.*; -import javax.xml.validation.TypeInfoProvider; import java.lang.reflect.Field; -import java.lang.reflect.Type; import java.util.*; +import java.util.concurrent.ExecutionException; import java.util.logging.Level; import java.util.stream.Collectors; @@ -58,34 +57,48 @@ public class IgniteClientDbCache implements IgniteICache { private static final int RESULT_THRESHOLD_SIZE = 100000; private ClientCache cache; private final IgniteClientTransaction transaction; + private class SqlField { + public String name; + public String type; + } + private Map sqlFields = null; + private boolean checkedForSqlFields = false; - public IgniteClientDbCache(IgniteClient igniteClient, String name) { + public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { + String className = v.getName(); try { - Map sqlFields = getSqlFields(); - if (!sqlFields.isEmpty()) { - this.cache = getOrCreateIndexedCache(igniteClient, sqlFields, name); + if (checkedForSqlFields == false) { + checkedForSqlFields = true; + extractSqlFields(className); + if (sqlFields != null && sqlFields.size() != 0) { + ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); + clientCacheConfig.setName(className); + this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); + } } if (this.cache == null) - this.cache = igniteClient.getOrCreateCache(name); + this.cache = igniteClient.getOrCreateCache(className); - logger.log(Level.INFO, "Cache " + name + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); + logger.log(Level.INFO, "Cache " + className + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } catch (ClientException e) { - logger.log(Level.WARNING, "Create cache for client " + name + " failed:" + e.getMessage()); + logger.log(Level.WARNING, "Create cache for client " + className + " failed:" + e.getMessage()); + logger.log(Level.WARNING, "Create cache for client " + className + " failed:" + e.getMessage()); } - Assert.notNull(this.cache, "Create cache for client " + name + "failed"); + Assert.notNull(this.cache, "Create cache for client " + className + "failed"); this.transaction = new IgniteClientTransaction(igniteClient); } - public IgniteClientDbCache(IgniteClient igniteClient, CacheConfiguration cacheConfig) { + public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfiguration cacheConfig) { try { + String className = v.getName(); ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); clientCacheConfig.setName(cacheConfig.getName()); clientCacheConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); logger.log(Level.INFO, "Getting or creating cache " + clientCacheConfig.getName() + " AtomicityMode is " + clientCacheConfig.getAtomicityMode()); - Map sqlFields = getSqlFields(); - if (!sqlFields.isEmpty()) { - this.cache = getOrCreateIndexedCache(igniteClient, sqlFields, clientCacheConfig); + extractSqlFields(className); + if (sqlFields != null) { + this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); } if (this.cache == null) this.cache = igniteClient.getOrCreateCache(clientCacheConfig); @@ -99,10 +112,21 @@ public IgniteClientDbCache(IgniteClient igniteClient, CacheConfiguration cacheCo this.transaction = new IgniteClientTransaction(igniteClient); } - public IgniteClientDbCache(IgniteClient igniteClient, String name, ExpiryPolicy ep) { + public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name, ExpiryPolicy ep) { try { - this.cache = igniteClient.getOrCreateCache(name).withExpirePolicy(ep); - logger.log(Level.INFO, "Cache " + name + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); + if (checkedForSqlFields == false) { + checkedForSqlFields = true; + extractSqlFields(v.getName()); + if (sqlFields != null) { + ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); + clientCacheConfig.setName(name); + getOrCreateIndexedCache(igniteClient, v.getName(), clientCacheConfig, ep); + } + } + if (this.cache == null) { + this.cache = igniteClient.getOrCreateCache(name).withExpirePolicy(ep); + logger.log(Level.INFO, "Cache " + name + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); + } } catch (ClientException e) { logger.log(Level.WARNING, "Create cache for client " + name + " failed:" + e.getMessage()); } @@ -111,87 +135,6 @@ public IgniteClientDbCache(IgniteClient igniteClient, String name, ExpiryPolicy this.transaction = new IgniteClientTransaction(igniteClient); } - /** - * - * @param igniteClient - * @param name - * If the class has QuerySqlField annotations, add query entry fields and indexes. - */ - private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Map sqlFields, String name) { - logger.log(Level.INFO, "Creating cache " + name + " with index"); - ClientCacheConfiguration ccConfig = new ClientCacheConfiguration(); - ccConfig.setName(name); - return getOrCreateIndexedCacheInternal(igniteClient, sqlFields, ccConfig); - } - - private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Map sqlFields, ClientCacheConfiguration cacheConfig) { - logger.log(Level.INFO, "Creating cache " + cacheConfig.getName() + " with index"); - ClientCacheConfiguration ccConfig = new ClientCacheConfiguration(); - ccConfig.setName(cacheConfig.getName()); - ccConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); - return getOrCreateIndexedCacheInternal(igniteClient, sqlFields, ccConfig); - } - - - private Map getSqlFields() { - Map sqlFields = new HashMap<>(); - Type t = getClass(); - logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + t.getTypeName()); - Class type = (Class) (new TypeToken(getClass()){}.getType()); - logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + type.getName()); - // go through all fields but pick the very first one, for now, - // make it work for multiple fileds later. - for (Field f : type.getDeclaredFields()) { - QuerySqlField annot = f.getAnnotation(QuerySqlField.class); - if (annot.notNull()) { - logger.log(Level.INFO, "annotation: " + annot.toString()); - if (annot.index()) { - if (sqlFields.keySet().isEmpty()) - sqlFields.put("index", annot.name()); - } - else if (sqlFields.values().isEmpty()) - sqlFields.put("value", annot.name()); - } - } - logger.log(Level.INFO, "Found " + sqlFields.size() + " sqlFields"); - return sqlFields; - } - - private ClientCache getOrCreateIndexedCacheInternal(IgniteClient igniteClient, Map sqlFields, ClientCacheConfiguration cachConfig) { - - QueryEntity qryEnt = new QueryEntity(); - String idxFld = sqlFields.get("index"); - String valFld = sqlFields.get("value"); - - Class type = (Class) (new TypeToken(getClass()){}.getType()); - String className = type.getName(); - logger.log(Level.INFO, "IndexedCache: " + cachConfig.getName() + " IDX " + idxFld + " VAL " + valFld + " TYP " + className); - try { - qryEnt.setKeyType(String.valueOf(type.getField(idxFld))); - qryEnt.setValueType(type.getClass().getName()); - qryEnt.addQueryField(idxFld, String.valueOf(type.getField(idxFld)), null); - qryEnt.addQueryField(valFld, String.valueOf(type.getField(valFld)), null); - QueryIndex qryIndex = new QueryIndex(idxFld); - qryEnt.setIndexes(Collections.singleton(qryIndex)); - } - catch (Exception e) { - logger.log(Level.WARNING, "Failed to create index on cache: " + cachConfig.getName() + ": " + e.getMessage()); - return null; - } - - cachConfig.setQueryEntities(qryEnt); - - // also make not of this cache, somewhere, somehow? - // have a static cache? - ClientCache cache = igniteClient.getOrCreateCache(cachConfig); - - String result = cache == null ? "FAILED" : "WORKED"; - - logger.log(Level.INFO, "Creating index on " + cachConfig.getName() + " " + result); - - return cache; - } - @Override public V get(K key) throws CacheException { @@ -263,10 +206,9 @@ public boolean remove(K key) throws CacheException { @Override public V get(Map filterParams) throws CacheException { - Map sqlFields = checkForSqlFieldsQuery(filterParams); - if (sqlFields != null) - return getSqlFields(sqlFields, filterParams); - + if (checkForSqlFieldsQuery(filterParams) == true) { + return getBySqlFields(filterParams); + } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); return get(predicate); } @@ -295,9 +237,9 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE @Override public Map getAll(Map filterParams) throws CacheException { - Map sqlFields = checkForSqlFieldsQuery(filterParams); - if (sqlFields != null) - return getSqlFieldsAll(sqlFields, filterParams); + if (checkForSqlFieldsQuery(filterParams) == true) { + return getBySqlFieldsAll(filterParams); + } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); return getAll(predicate); } @@ -333,19 +275,9 @@ public Transaction getTransaction() { return transaction; } - private Map checkForSqlFieldsQuery(Map filterParams) { - Map sqlFields = new HashMap<>(); - Class type = (Class) (new TypeToken(getClass()) {}.getType()); - for (Field f : type.getDeclaredFields()) { - QuerySqlField annot = f.getAnnotation(QuerySqlField.class); - if (annot.notNull()) { - if (annot.index()) - sqlFields.put("key", annot.name()); - else - sqlFields.put("value", annot.name()); - } - } - + private boolean checkForSqlFieldsQuery(Map filterParams) { + if (checkedForSqlFields == true && sqlFields == null) + return false; /* * There must be exactly two sqlfileds, one for the index lookup * and the other for the _VAL (V) field in the Class declaration. @@ -354,15 +286,15 @@ private Map checkForSqlFieldsQuery(Map filterP * If these conditions are true, run SQLFieldsQuery otherwise, ScanQuery. */ if (sqlFields.size() == 2 && filterParams.size() == 1 && sqlFields.containsKey(filterParams.keySet())) { - return sqlFields; + return true; } - return null; + return false; } - private V getSqlFields(Map sqlFields, Map filterParams) { + private V getBySqlFields(Map filterParams) { try { - Map result = runSQLFieldsQuery(sqlFields, filterParams); + Map result = runSQLFieldsQuery(filterParams); if (result == null || result.isEmpty()) return null; return (V) result.get(0); @@ -372,9 +304,9 @@ private V getSqlFields(Map sqlFields, Map Map getSqlFieldsAll(Map sqlFields, Map filterParams) { + public Map getBySqlFieldsAll(Map filterParams) { try { - Map values = runSQLFieldsQuery(sqlFields, filterParams); + Map values = runSQLFieldsQuery(filterParams); return values; } catch (Exception e) { @@ -387,9 +319,9 @@ public Map getSqlFieldsAll(Map sqlFields, Map Map runSQLFieldsQuery(Map sqlFields, Map filterParams) throws CacheException { + private Map runSQLFieldsQuery(Map filterParams) throws CacheException { - String sql = buildSqlFieldsQuery(sqlFields, filterParams); + String sql = buildSqlFieldsQuery(filterParams); SqlFieldsQuery query = new SqlFieldsQuery(sql); Map results = new HashMap<>(); @@ -405,9 +337,9 @@ private Map runSQLFieldsQuery(Map sqlFields, Map< } } - private String buildSqlFieldsQuery(Map sqlFields, Map filterParams){ + private String buildSqlFieldsQuery(Map filterParams){ StringBuilder sb = new StringBuilder("select "); - sb.append(sqlFields.get("value")).append(" from \""); + sb.append(sqlFields.get("value").name).append(" from \""); sb.append(cache.getConfiguration().getName()).append("\""); sb.append(" where "); sb.append(filterParams.keySet()).append(" = "); @@ -415,4 +347,117 @@ private String buildSqlFieldsQuery(Map sqlFields, Map getOrCreateIndexedCache(IgniteClient igniteClient, String className, ClientCacheConfiguration cacheConfig, ExpiryPolicy ep) { + String cacheName = cacheConfig.getName(); + logger.log(Level.INFO, "Creating cache " + cacheName + " with index"); + + SqlField idxFld = sqlFields.get("index"); + SqlField valFld = sqlFields.get("value"); + Class v = null; + try { + v = Class.forName(className); + } catch (Exception e) { + logger.log(Level.INFO, "Failed to get class for " + className); + return null; + } + QueryEntity qryEnt = null; + + try { + String tblName = CommonUtil.getSimpleFromCanonicalName(valFld.name); + qryEnt = new QueryEntity(idxFld.type, valFld.name).setTableName(tblName); + logger.log(Level.INFO, "Setting table " + tblName + " for cache " + className); + } catch (Exception e) { + logger.log(Level.INFO, "Failed to get type for index field " + idxFld.name); + return null; + } + try { + qryEnt.setKeyFieldName(idxFld.name); + qryEnt.setValueType(idxFld.type); + qryEnt.addQueryField(idxFld.name, idxFld.type, null); + qryEnt.addQueryField(valFld.name, valFld.type, null); + QueryIndex qryIndex = new QueryIndex(idxFld.name); + qryEnt.setIndexes(Collections.singleton(qryIndex)); + } + catch (Exception e) { + logger.log(Level.WARNING, "Failed to create index on cache: " + cacheName + ": " + e.getMessage()); + return null; + } + + cacheConfig.setQueryEntities(qryEnt); + String schName = CommonUtil.getSchemaNameForCacheClass(v.getName()); + logger.log(Level.INFO, "Setting schema name " + schName + " for cahce " + cacheName); + cacheConfig.setSqlSchema(schName); + + // also make not of this cache, somewhere, somehow? + // have a static cache? + ClientCache cache = null; + + if (ep != null) { + cache = igniteClient.getOrCreateCache(cacheConfig).withExpirePolicy(ep); + } + else { + cache = igniteClient.getOrCreateCache(cacheConfig); + } + + String result = cache == null ? "FAILED" : "WORKED"; + + logger.log(Level.INFO, "Creating index on " + cacheName + " " + result); + + return cache; + } + + + private void extractSqlFields(String className) { + Map localFields = new HashMap<>(); + logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + className); + Field[] fields = null; + try { + Class v = Class.forName(className); + + fields = ControllerUtil.getAllDeclaredFields(v); + + // go through all fields but pick the very first one, for now, + // make it work for multiple fields later. + for (Field f : fields) { + QuerySqlField annot = f.getAnnotation(QuerySqlField.class); + if (annot != null) { + logger.log(Level.INFO, "Found for " + f.getName() + " annotation: " + annot.toString()); + if (annot.index()) { + if (localFields.keySet().isEmpty()) { + SqlField sqlField = new SqlField(); + sqlField.name = f.getName(); + sqlField.type = f.getType().getTypeName(); + logger.log(Level.INFO, "Adding index for " + sqlField.name + " with type " + sqlField.type); + localFields.put("index", sqlField); + } + } else { + SqlField sqlField = new SqlField(); + sqlField.name = f.getName(); + sqlField.type = f.getType().getTypeName(); + logger.log(Level.INFO, "Adding value for " + sqlField.name + " with type " + sqlField.type); + sqlFields.put("value", sqlField); + } + } + } + + if (localFields.isEmpty() == false && localFields.containsValue(className) == false) { + SqlField sqlField = new SqlField(); + sqlField.name = className; + sqlField.type = v.getTypeName(); + logger.log(Level.INFO, "Adding value for " + sqlField.name + " with type " + sqlField.type); + localFields.put("value", sqlField); + } + sqlFields = localFields; + logger.log(Level.INFO, "Found " + sqlFields.size() + " sqlFields"); + } catch (Exception e) { + // + } + } } \ No newline at end of file diff --git a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java index f7278f0bc..b250cd53c 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java +++ b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java @@ -103,7 +103,7 @@ public static boolean isNullOrEmpty(String string) { } /** - * Reture CacheConfiguration for transaction + * Return CacheConfiguration for transaction * @param cacheName input String * @return return cache configuration */ @@ -114,4 +114,24 @@ public static CacheConfiguration getCacheConfiguration(String cacheName) { cfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); return cfg; } -} + + /** + * Return a simple name of the class or member given a canonical name as string. + * @param canon + * @return simpleName + */ + + public static String getSimpleFromCanonicalName(String canon) { + int lastDot = canon.lastIndexOf("."); + return canon.substring(lastDot + 1); + } + + /** + * Replace all dots with underscore to make a schema name + * @param className + * @return + */ + public static String getSchemaNameForCacheClass(String className) { + return className.replaceAll("\\.", "_"); + } +} \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java index 80dbe2041..af6e46f91 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java @@ -44,7 +44,7 @@ public class Router extends CustomerResource { private String vpcDefaultRouteTableId; // store vpc_id - @QuerySqlField + @QuerySqlField(index = true) @JsonProperty("owner") private String owner; From 80214bd79dfd0ef9e0410c28624ebcd3ecf65ccc Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Thu, 28 Oct 2021 14:16:57 -0700 Subject: [PATCH 06/27] SQL can select but the value of key column is the literal string --- .../common/db/ignite/IgniteClientDbCache.java | 74 +++++++++++++------ 1 file changed, 51 insertions(+), 23 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 2338ac801..e5200b170 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -42,6 +42,7 @@ free of charge, to any person obtaining a copy of this software and associated d import org.apache.ignite.configuration.CacheConfiguration; import org.apache.ignite.lang.IgniteBiPredicate; import org.springframework.util.Assert; +import org.springframework.util.TypeUtils; import javax.cache.Cache; import javax.cache.expiry.ExpiryPolicy; @@ -55,6 +56,7 @@ public class IgniteClientDbCache implements IgniteICache { private static final Logger logger = LoggerFactory.getLogger(); private static final int RESULT_THRESHOLD_SIZE = 100000; + private static final String SQL_SCHEMA_NAME = "alcor"; private ClientCache cache; private final IgniteClientTransaction transaction; private class SqlField { @@ -67,7 +69,7 @@ private class SqlField { public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { String className = v.getName(); try { - if (checkedForSqlFields == false) { + if (!checkedForSqlFields) { checkedForSqlFields = true; extractSqlFields(className); if (sqlFields != null && sqlFields.size() != 0) { @@ -114,7 +116,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name, ExpiryPolicy ep) { try { - if (checkedForSqlFields == false) { + if (!checkedForSqlFields) { checkedForSqlFields = true; extractSqlFields(v.getName()); if (sqlFields != null) { @@ -206,7 +208,7 @@ public boolean remove(K key) throws CacheException { @Override public V get(Map filterParams) throws CacheException { - if (checkForSqlFieldsQuery(filterParams) == true) { + if (!checkForSqlFieldsQuery(filterParams)) { return getBySqlFields(filterParams); } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); @@ -237,7 +239,7 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE @Override public Map getAll(Map filterParams) throws CacheException { - if (checkForSqlFieldsQuery(filterParams) == true) { + if (!checkForSqlFieldsQuery(filterParams)) { return getBySqlFieldsAll(filterParams); } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); @@ -276,7 +278,7 @@ public Transaction getTransaction() { } private boolean checkForSqlFieldsQuery(Map filterParams) { - if (checkedForSqlFields == true && sqlFields == null) + if (checkedForSqlFields && (sqlFields == null || sqlFields.size() != 1)) return false; /* * There must be exactly two sqlfileds, one for the index lookup @@ -297,7 +299,7 @@ private V getBySqlFields(Map filterParams) { Map result = runSQLFieldsQuery(filterParams); if (result == null || result.isEmpty()) return null; - return (V) result.get(0); + return result.get(0); } catch (Exception e) { return null; @@ -306,10 +308,10 @@ private V getBySqlFields(Map filterParams) { public Map getBySqlFieldsAll(Map filterParams) { try { - Map values = runSQLFieldsQuery(filterParams); - return values; + return runSQLFieldsQuery(filterParams); } catch (Exception e) { + logger.log(Level.INFO, "getBySqlFieldsAll failed"); return null; } } @@ -337,14 +339,31 @@ private Map runSQLFieldsQuery(Map filterParams) } } - private String buildSqlFieldsQuery(Map filterParams){ + private String buildSqlFieldsQuery(Map filterParams) { + SqlField valFld = sqlFields.get("value"); + String valName = valFld.name; + String vaType = valFld.type; + Class v; + try { + v = Class.forName(vaType); + } catch (Exception e) { + logger.log(Level.INFO, "Failed to find type of " + vaType); + return null; + } + StringBuilder sb = new StringBuilder("select "); - sb.append(sqlFields.get("value").name).append(" from \""); - sb.append(cache.getConfiguration().getName()).append("\""); + sb.append(valName).append(" from \""); + sb.append(cache.getConfiguration().getSqlSchema()).append("\"."); + sb.append(cache.getConfiguration().getName()); sb.append(" where "); sb.append(filterParams.keySet()).append(" = "); + if (TypeUtils.isAssignable(v, String.class)) { + sb.append("'"); + } sb.append(filterParams.values()); - + if (TypeUtils.isAssignable(v, String.class)) { + sb.append("'"); + } return sb.toString(); } @@ -360,29 +379,37 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str SqlField idxFld = sqlFields.get("index"); SqlField valFld = sqlFields.get("value"); - Class v = null; + Class v; try { v = Class.forName(className); } catch (Exception e) { logger.log(Level.INFO, "Failed to get class for " + className); return null; } - QueryEntity qryEnt = null; + QueryEntity qryEnt; try { - String tblName = CommonUtil.getSimpleFromCanonicalName(valFld.name); - qryEnt = new QueryEntity(idxFld.type, valFld.name).setTableName(tblName); + String tblName = CommonUtil.getSimpleFromCanonicalName(cacheName); + qryEnt = new QueryEntity(idxFld.type, className).setTableName(tblName); + logger.log(Level.INFO, "QueryEntity = " + qryEnt); logger.log(Level.INFO, "Setting table " + tblName + " for cache " + className); } catch (Exception e) { logger.log(Level.INFO, "Failed to get type for index field " + idxFld.name); return null; } try { - qryEnt.setKeyFieldName(idxFld.name); - qryEnt.setValueType(idxFld.type); + // qryEnt.setKeyFieldName(idxFld.name); + qryEnt.setKeyType(idxFld.type); + logger.log(Level.INFO, "QE: setKeyType = " + idxFld.type); + qryEnt.setValueType(className); + logger.log(Level.INFO, "QE: setValueType = " + cacheName); qryEnt.addQueryField(idxFld.name, idxFld.type, null); + logger.log(Level.INFO, "AQF: name = " + idxFld.name + ", type = " + idxFld.type); + qryEnt.addQueryField(valFld.name, valFld.type, null); + logger.log(Level.INFO, "AQF: name = " + valFld.name + ", type = " + valFld.type); QueryIndex qryIndex = new QueryIndex(idxFld.name); + logger.log(Level.INFO, "QI: name = " + idxFld.name); qryEnt.setIndexes(Collections.singleton(qryIndex)); } catch (Exception e) { @@ -391,13 +418,14 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str } cacheConfig.setQueryEntities(qryEnt); - String schName = CommonUtil.getSchemaNameForCacheClass(v.getName()); - logger.log(Level.INFO, "Setting schema name " + schName + " for cahce " + cacheName); - cacheConfig.setSqlSchema(schName); + // String schName = CommonUtil.getSchemaNameForCacheClass(v.getName()); + logger.log(Level.INFO, "Setting schema name " + SQL_SCHEMA_NAME + " for cahce " + cacheName); + cacheConfig.setSqlSchema(SQL_SCHEMA_NAME); + logger.log(Level.INFO, "cacheConfig = " + cacheConfig.toString()); // also make not of this cache, somewhere, somehow? // have a static cache? - ClientCache cache = null; + ClientCache cache; if (ep != null) { cache = igniteClient.getOrCreateCache(cacheConfig).withExpirePolicy(ep); @@ -447,7 +475,7 @@ private void extractSqlFields(String className) { } } - if (localFields.isEmpty() == false && localFields.containsValue(className) == false) { + if (!(localFields.isEmpty() || localFields.containsValue(className))) { SqlField sqlField = new SqlField(); sqlField.name = className; sqlField.type = v.getTypeName(); From 3bb815b211b2d870f1f794d41dc1ded2473fc13b Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Thu, 28 Oct 2021 14:48:32 -0700 Subject: [PATCH 07/27] SQL Query works now --- .../common/db/ignite/IgniteClientDbCache.java | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index e5200b170..b6623c12c 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -390,7 +390,7 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str try { String tblName = CommonUtil.getSimpleFromCanonicalName(cacheName); - qryEnt = new QueryEntity(idxFld.type, className).setTableName(tblName); + qryEnt = new QueryEntity().setTableName(tblName); logger.log(Level.INFO, "QueryEntity = " + qryEnt); logger.log(Level.INFO, "Setting table " + tblName + " for cache " + className); } catch (Exception e) { @@ -400,16 +400,19 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str try { // qryEnt.setKeyFieldName(idxFld.name); qryEnt.setKeyType(idxFld.type); - logger.log(Level.INFO, "QE: setKeyType = " + idxFld.type); qryEnt.setValueType(className); - logger.log(Level.INFO, "QE: setValueType = " + cacheName); + qryEnt.setKeyFieldName(idxFld.name); + qryEnt.setValueFieldName(valFld.name); qryEnt.addQueryField(idxFld.name, idxFld.type, null); - logger.log(Level.INFO, "AQF: name = " + idxFld.name + ", type = " + idxFld.type); + logger.log(Level.INFO, "QE: " + qryEnt); + // qryEnt.setKeyType(valFld.name); + // qryEnt.setValueType(valFld.type); qryEnt.addQueryField(valFld.name, valFld.type, null); - logger.log(Level.INFO, "AQF: name = " + valFld.name + ", type = " + valFld.type); + logger.log(Level.INFO, "QE: " + qryEnt); + QueryIndex qryIndex = new QueryIndex(idxFld.name); - logger.log(Level.INFO, "QI: name = " + idxFld.name); + logger.log(Level.INFO, "QI: " + qryIndex); qryEnt.setIndexes(Collections.singleton(qryIndex)); } catch (Exception e) { From 05cc8573cc695b03d48101b8af768b32fccd3e5b Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Thu, 28 Oct 2021 18:05:26 -0700 Subject: [PATCH 08/27] SQL Query works (one key column, one value column only) --- .../common/db/ignite/IgniteClientDbCache.java | 31 +++++++++++-------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index b6623c12c..60de4f269 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -208,7 +208,7 @@ public boolean remove(K key) throws CacheException { @Override public V get(Map filterParams) throws CacheException { - if (!checkForSqlFieldsQuery(filterParams)) { + if (checkForSqlFieldsQuery(filterParams)) { return getBySqlFields(filterParams); } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); @@ -239,7 +239,7 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE @Override public Map getAll(Map filterParams) throws CacheException { - if (!checkForSqlFieldsQuery(filterParams)) { + if (checkForSqlFieldsQuery(filterParams)) { return getBySqlFieldsAll(filterParams); } IgniteBiPredicate predicate = MapPredicate.getInstance(filterParams); @@ -278,7 +278,7 @@ public Transaction getTransaction() { } private boolean checkForSqlFieldsQuery(Map filterParams) { - if (checkedForSqlFields && (sqlFields == null || sqlFields.size() != 1)) + if (checkedForSqlFields && (sqlFields == null || sqlFields.size() == 0)) return false; /* * There must be exactly two sqlfileds, one for the index lookup @@ -287,7 +287,7 @@ private boolean checkForSqlFieldsQuery(Map filterParams) { * The entry in the filterparams should be an indexed field. * If these conditions are true, run SQLFieldsQuery otherwise, ScanQuery. */ - if (sqlFields.size() == 2 && filterParams.size() == 1 && sqlFields.containsKey(filterParams.keySet())) { + if (sqlFields.size() == 2 && filterParams.size() == 1 && filterParams.containsKey(sqlFields.get("index").name)) { return true; } @@ -343,6 +343,7 @@ private String buildSqlFieldsQuery(Map filterParams) { SqlField valFld = sqlFields.get("value"); String valName = valFld.name; String vaType = valFld.type; + String keyFldName = sqlFields.get("index").name; Class v; try { v = Class.forName(vaType); @@ -351,19 +352,23 @@ private String buildSqlFieldsQuery(Map filterParams) { return null; } + boolean needQuotes = false; + String keyType = sqlFields.get("index").type; + if (this.sqlFields.get("index").type.equals("java.lang.String")) + needQuotes = true; + StringBuilder sb = new StringBuilder("select "); - sb.append(valName).append(" from \""); - sb.append(cache.getConfiguration().getSqlSchema()).append("\"."); - sb.append(cache.getConfiguration().getName()); + sb.append(sqlFields.get("index").name).append(", "); + sb.append(CommonUtil.getSimpleFromCanonicalName(valName)).append(" from "); + sb.append(SQL_SCHEMA_NAME).append("."); + sb.append(cache.getConfiguration().getQueryEntities()[0].getTableName()); sb.append(" where "); - sb.append(filterParams.keySet()).append(" = "); - if (TypeUtils.isAssignable(v, String.class)) { + sb.append(sqlFields.get("index").name).append(" = "); + if (needQuotes) sb.append("'"); - } - sb.append(filterParams.values()); - if (TypeUtils.isAssignable(v, String.class)) { + sb.append(filterParams.get(keyFldName)[0]); + if (needQuotes) sb.append("'"); - } return sb.toString(); } From a40db7eda0f4b5b95b5028a72153a146917cc73a Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Thu, 4 Nov 2021 17:15:41 -0700 Subject: [PATCH 09/27] Fix SQL result from wrong column problem --- .../common/db/ignite/IgniteClientDbCache.java | 178 +++++++++--------- .../alcor/common/utils/CommonUtil.java | 7 +- .../NetworkConfigManagerApplication.java | 5 + .../alcor/web/entity/node/NodeInfo.java | 5 +- 4 files changed, 108 insertions(+), 87 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 60de4f269..20aea649d 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -24,8 +24,6 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.logging.LoggerFactory; import com.futurewei.alcor.common.utils.CommonUtil; import com.futurewei.alcor.common.utils.ControllerUtil; -import com.google.common.reflect.TypeToken; -import io.netty.util.concurrent.EventExecutorChooserFactory; import org.apache.ignite.binary.BinaryObject; import org.apache.ignite.cache.CachePeekMode; import org.apache.ignite.cache.QueryEntity; @@ -42,28 +40,30 @@ free of charge, to any person obtaining a copy of this software and associated d import org.apache.ignite.configuration.CacheConfiguration; import org.apache.ignite.lang.IgniteBiPredicate; import org.springframework.util.Assert; -import org.springframework.util.TypeUtils; import javax.cache.Cache; import javax.cache.expiry.ExpiryPolicy; import java.lang.reflect.Field; +import java.sql.Array; import java.util.*; -import java.util.concurrent.ExecutionException; import java.util.logging.Level; import java.util.stream.Collectors; +import static org.apache.ignite.internal.util.lang.GridFunc.asList; + public class IgniteClientDbCache implements IgniteICache { private static final Logger logger = LoggerFactory.getLogger(); - + private static final String NON_SCALAR_ROWSET = "more than one rows found!"; private static final int RESULT_THRESHOLD_SIZE = 100000; private static final String SQL_SCHEMA_NAME = "alcor"; private ClientCache cache; private final IgniteClientTransaction transaction; - private class SqlField { - public String name; + private static class SqlField { public String type; + public boolean isIndexed; } - private Map sqlFields = null; + private Map sqlFields = null; // needed for index creation and querying + private List sqlColumns = null; // needed for constructing select list private boolean checkedForSqlFields = false; public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { @@ -74,7 +74,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { extractSqlFields(className); if (sqlFields != null && sqlFields.size() != 0) { ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); - clientCacheConfig.setName(className); + clientCacheConfig.setName(name); this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); } } @@ -221,7 +221,7 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE cache.withKeepBinary().query(ScanQueryBuilder.newScanQuery(igniteBiPredicate)); List> result = cursor.getAll(); if(result.size() > 1){ - throw new CacheException("more than one rows found!"); + throw new CacheException(NON_SCALAR_ROWSET); } if(result.isEmpty()){ @@ -280,30 +280,38 @@ public Transaction getTransaction() { private boolean checkForSqlFieldsQuery(Map filterParams) { if (checkedForSqlFields && (sqlFields == null || sqlFields.size() == 0)) return false; + /* - * There must be exactly two sqlfileds, one for the index lookup - * and the other for the _VAL (V) field in the Class declaration. - * There can only one be exactly one field in queryParams for now. - * The entry in the filterparams should be an indexed field. - * If these conditions are true, run SQLFieldsQuery otherwise, ScanQuery. + * All filterParams must be found in sqlfields as "index". */ - if (sqlFields.size() == 2 && filterParams.size() == 1 && filterParams.containsKey(sqlFields.get("index").name)) { - return true; + int idxCount = 0; + for (String f : filterParams.keySet()) { + if (sqlFields.containsKey(f)) + ++idxCount; } - return false; + return idxCount == filterParams.size(); } - private V getBySqlFields(Map filterParams) { + private V getBySqlFields(Map filterParams) throws CacheException { try { Map result = runSQLFieldsQuery(filterParams); if (result == null || result.isEmpty()) return null; + if(result.size() > 1) { + throw new CacheException(NON_SCALAR_ROWSET); + } return result.get(0); } + catch (CacheException ce) { + if (ce.getMessage().equals(NON_SCALAR_ROWSET)) + throw ce; + } catch (Exception e) { - return null; + logger.log(Level.WARNING, "runSQLFieldsQuery exception : " + e.getMessage()); } + + return null; } public Map getBySqlFieldsAll(Map filterParams) { @@ -311,7 +319,7 @@ public Map getBySqlFieldsAll(Map filterParams) return runSQLFieldsQuery(filterParams); } catch (Exception e) { - logger.log(Level.INFO, "getBySqlFieldsAll failed"); + logger.log(Level.INFO, "runSQLFieldsQuery exception: " + e.getMessage()); return null; } } @@ -340,35 +348,35 @@ private Map runSQLFieldsQuery(Map filterParams) } private String buildSqlFieldsQuery(Map filterParams) { - SqlField valFld = sqlFields.get("value"); - String valName = valFld.name; - String vaType = valFld.type; - String keyFldName = sqlFields.get("index").name; + SqlField valFld = sqlFields.get("$value"); Class v; try { - v = Class.forName(vaType); + v = Class.forName(valFld.type); } catch (Exception e) { - logger.log(Level.INFO, "Failed to find type of " + vaType); + logger.log(Level.INFO, "Failed to find type of " + valFld.type); return null; } - boolean needQuotes = false; - String keyType = sqlFields.get("index").type; - if (this.sqlFields.get("index").type.equals("java.lang.String")) - needQuotes = true; - - StringBuilder sb = new StringBuilder("select "); - sb.append(sqlFields.get("index").name).append(", "); - sb.append(CommonUtil.getSimpleFromCanonicalName(valName)).append(" from "); - sb.append(SQL_SCHEMA_NAME).append("."); - sb.append(cache.getConfiguration().getQueryEntities()[0].getTableName()); - sb.append(" where "); - sb.append(sqlFields.get("index").name).append(" = "); - if (needQuotes) - sb.append("'"); - sb.append(filterParams.get(keyFldName)[0]); - if (needQuotes) - sb.append("'"); + StringBuilder sb = new StringBuilder("select _key, _val from " + SQL_SCHEMA_NAME + + "." + cache.getConfiguration().getQueryEntities()[0].getTableName() + " where "); + boolean needAnd = false; + for (String c : sqlColumns) { + if (!sqlFields.get(c).isIndexed) + continue; + SqlField f = sqlFields.get(c); + if (needAnd) + sb.append(" and "); + needAnd = true; + + sb.append(c).append(" = "); + boolean needQuotes = f.type.equals("java.lang.String"); + sb.append("'"); + sb.append(filterParams.get(c)[0]); + if (needQuotes) + sb.append("'"); + } + logger.log(Level.INFO, "SQL = " + sb.toString()); + return sb.toString(); } @@ -379,11 +387,10 @@ private String buildSqlFieldsQuery(Map filterParams) { * If the class has QuerySqlField annotations, add query entry fields and indexes. */ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, String className, ClientCacheConfiguration cacheConfig, ExpiryPolicy ep) { - String cacheName = cacheConfig.getName(); + // TODO: Rename getSchemaNameForCacheClass to something like getSqlNameFromJavaName + String cacheName = CommonUtil.getSchemaNameForCacheClass(cacheConfig.getName()); logger.log(Level.INFO, "Creating cache " + cacheName + " with index"); - SqlField idxFld = sqlFields.get("index"); - SqlField valFld = sqlFields.get("value"); Class v; try { v = Class.forName(className); @@ -391,34 +398,37 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str logger.log(Level.INFO, "Failed to get class for " + className); return null; } - QueryEntity qryEnt; + QueryEntity qryEnt = new QueryEntity().setTableName(cacheName); + qryEnt.setValueType(className); + + LinkedHashMap qryFields = new LinkedHashMap<>(); + ArrayList idxFields = new ArrayList<>(); try { - String tblName = CommonUtil.getSimpleFromCanonicalName(cacheName); - qryEnt = new QueryEntity().setTableName(tblName); - logger.log(Level.INFO, "QueryEntity = " + qryEnt); - logger.log(Level.INFO, "Setting table " + tblName + " for cache " + className); - } catch (Exception e) { - logger.log(Level.INFO, "Failed to get type for index field " + idxFld.name); - return null; - } - try { - // qryEnt.setKeyFieldName(idxFld.name); - qryEnt.setKeyType(idxFld.type); - qryEnt.setValueType(className); - qryEnt.setKeyFieldName(idxFld.name); - qryEnt.setValueFieldName(valFld.name); - qryEnt.addQueryField(idxFld.name, idxFld.type, null); + for (String c : sqlColumns) { + if (c.equals("$value")) + continue; + // qryEnt.setKeyFieldName(idxFld.name); + // qryEnt.setKeyType(idxFld.name); + // qryEnt.setValueType(className); + SqlField f = sqlFields.get(c); + logger.log(Level.INFO, "getOrCreateIndexedCache: Adding " + c + " to query fields " + " in " + cacheName); + qryFields.put(c, f.type); + if (f.isIndexed) { + logger.log(Level.INFO, "getOrCreateIndexedCache: Adding " + c + " to index fields " + " in " + cacheName); + idxFields.add(new QueryIndex(c)); + } + } + + qryEnt.setFields(qryFields); + logger.log(Level.INFO, "QE: " + qryEnt); // qryEnt.setKeyType(valFld.name); // qryEnt.setValueType(valFld.type); - qryEnt.addQueryField(valFld.name, valFld.type, null); - logger.log(Level.INFO, "QE: " + qryEnt); - QueryIndex qryIndex = new QueryIndex(idxFld.name); - logger.log(Level.INFO, "QI: " + qryIndex); - qryEnt.setIndexes(Collections.singleton(qryIndex)); + qryEnt.setIndexes(idxFields); + logger.log(Level.INFO, "cache : " + cacheName + ", QE = " + qryEnt.toString()); } catch (Exception e) { logger.log(Level.WARNING, "Failed to create index on cache: " + cacheName + ": " + e.getMessage()); @@ -427,7 +437,7 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str cacheConfig.setQueryEntities(qryEnt); // String schName = CommonUtil.getSchemaNameForCacheClass(v.getName()); - logger.log(Level.INFO, "Setting schema name " + SQL_SCHEMA_NAME + " for cahce " + cacheName); + logger.log(Level.INFO, "Setting schema name " + SQL_SCHEMA_NAME + " for cache " + cacheName); cacheConfig.setSqlSchema(SQL_SCHEMA_NAME); logger.log(Level.INFO, "cacheConfig = " + cacheConfig.toString()); @@ -452,6 +462,7 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str private void extractSqlFields(String className) { Map localFields = new HashMap<>(); + List localColumns = new ArrayList<>(); logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + className); Field[] fields = null; try { @@ -459,39 +470,38 @@ private void extractSqlFields(String className) { fields = ControllerUtil.getAllDeclaredFields(v); - // go through all fields but pick the very first one, for now, - // make it work for multiple fields later. for (Field f : fields) { QuerySqlField annot = f.getAnnotation(QuerySqlField.class); if (annot != null) { - logger.log(Level.INFO, "Found for " + f.getName() + " annotation: " + annot.toString()); + logger.log(Level.INFO, "QuerySqlField Found for " + f.getName() + " annotation: " + annot.toString()); if (annot.index()) { - if (localFields.keySet().isEmpty()) { SqlField sqlField = new SqlField(); - sqlField.name = f.getName(); + sqlField.isIndexed = true; sqlField.type = f.getType().getTypeName(); - logger.log(Level.INFO, "Adding index for " + sqlField.name + " with type " + sqlField.type); - localFields.put("index", sqlField); - } + logger.log(Level.INFO, "QuerySqlField Adding index for " + f.getName() + " with type " + sqlField.type); + localFields.put(f.getName(), sqlField); } else { SqlField sqlField = new SqlField(); - sqlField.name = f.getName(); + sqlField.isIndexed = false; sqlField.type = f.getType().getTypeName(); - logger.log(Level.INFO, "Adding value for " + sqlField.name + " with type " + sqlField.type); - sqlFields.put("value", sqlField); + logger.log(Level.INFO, "QuerySqlField Adding value for " + f.getName() + " with type " + sqlField.type); + localFields.put(f.getName(), sqlField); } + + localColumns.add(f.getName()); } } if (!(localFields.isEmpty() || localFields.containsValue(className))) { SqlField sqlField = new SqlField(); - sqlField.name = className; + sqlField.isIndexed = false; sqlField.type = v.getTypeName(); - logger.log(Level.INFO, "Adding value for " + sqlField.name + " with type " + sqlField.type); - localFields.put("value", sqlField); + logger.log(Level.INFO, "QuerySqlField Adding value for _VAL with type " + sqlField.type); + localFields.put("$value", sqlField); } sqlFields = localFields; - logger.log(Level.INFO, "Found " + sqlFields.size() + " sqlFields"); + sqlColumns = localColumns; + logger.log(Level.INFO, "QuerySqlField Found " + sqlFields.size() + " sqlFields"); } catch (Exception e) { // } diff --git a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java index b250cd53c..6e79f1a7b 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java +++ b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java @@ -123,7 +123,10 @@ public static CacheConfiguration getCacheConfiguration(String cacheName) { public static String getSimpleFromCanonicalName(String canon) { int lastDot = canon.lastIndexOf("."); - return canon.substring(lastDot + 1); + if (lastDot != -1) + return canon.substring(lastDot + 1); + else + return canon; } /** @@ -134,4 +137,4 @@ public static String getSimpleFromCanonicalName(String canon) { public static String getSchemaNameForCacheClass(String className) { return className.replaceAll("\\.", "_"); } -} \ No newline at end of file +} diff --git a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java index 3f9398efe..ff0270dbe 100644 --- a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java +++ b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java @@ -37,12 +37,17 @@ public class NetworkConfigManagerApplication { @PostConstruct public void instantiateGrpcServer(){ + return; + /** + * Return so that rest API endpoint can work + * try { networkConfigServer.start(); networkConfigServer.blockUntilShutdown(); } catch (Exception e) { e.printStackTrace(); } + */ } public static void main(String[] args) { diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java index 644dc95e0..82643f003 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java @@ -18,6 +18,7 @@ free of charge, to any person obtaining a copy of this software and associated d import com.fasterxml.jackson.annotation.JsonAutoDetect; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.Data; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -32,9 +33,11 @@ public class NodeInfo implements Serializable { private static final Logger logger = LoggerFactory.getLogger(NodeInfo.class); @JsonProperty("node_id") + @QuerySqlField(index = true) private String id; @JsonProperty("node_name") + @QuerySqlField(index = true) private String name; @JsonProperty("local_ip") @@ -180,4 +183,4 @@ public void setHostDvrMac(String hostDvrMac) { public String getNcmUri() { return ncm_uri; } public void setNcmUri(String uri) { ncm_uri = uri; } -} +} \ No newline at end of file From b68b0423eba0b7c8acefd5af28fac1a66e09628a Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 8 Nov 2021 10:55:30 -0800 Subject: [PATCH 10/27] fix filterkey matching bug --- .../futurewei/alcor/common/db/ignite/IgniteClientDbCache.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 20aea649d..22e8501fe 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -361,7 +361,7 @@ private String buildSqlFieldsQuery(Map filterParams) { "." + cache.getConfiguration().getQueryEntities()[0].getTableName() + " where "); boolean needAnd = false; for (String c : sqlColumns) { - if (!sqlFields.get(c).isIndexed) + if (!sqlFields.get(c).isIndexed || !filterParams.containsKey(c)) continue; SqlField f = sqlFields.get(c); if (needAnd) From 8703ec3d8ab3b594d8c5ba01737747796f93b78c Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 8 Nov 2021 11:35:15 -0800 Subject: [PATCH 11/27] Instrument Port, Subnet, Neighbor caches with SQL Fields --- .../com/futurewei/alcor/common/entity/CustomerResource.java | 4 +++- .../com/futurewei/alcor/portmanager/entity/PortIdSubnet.java | 4 +++- .../futurewei/alcor/web/entity/dataplane/NeighborInfo.java | 4 +++- .../java/com/futurewei/alcor/web/entity/port/PortEntity.java | 4 +++- 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java index 0c715c4b0..98555e79f 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java +++ b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java @@ -17,12 +17,14 @@ free of charge, to any person obtaining a copy of this software and associated d package com.futurewei.alcor.common.entity; import com.fasterxml.jackson.annotation.JsonProperty; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import java.util.Objects; public class CustomerResource extends Resource { @JsonProperty("project_id") + @QuerySqlField(index = true) private String projectId; @JsonProperty("tenant_id") @@ -110,4 +112,4 @@ public String toString() { ", description='" + description + '\'' + '}'; } -} +} \ No newline at end of file diff --git a/services/port_manager/src/main/java/com/futurewei/alcor/portmanager/entity/PortIdSubnet.java b/services/port_manager/src/main/java/com/futurewei/alcor/portmanager/entity/PortIdSubnet.java index 99e1690b6..986ac7789 100644 --- a/services/port_manager/src/main/java/com/futurewei/alcor/portmanager/entity/PortIdSubnet.java +++ b/services/port_manager/src/main/java/com/futurewei/alcor/portmanager/entity/PortIdSubnet.java @@ -16,11 +16,13 @@ free of charge, to any person obtaining a copy of this software and associated d package com.futurewei.alcor.portmanager.entity; import com.fasterxml.jackson.annotation.JsonProperty; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import java.util.Set; public class PortIdSubnet { @JsonProperty("subnet_id") + @QuerySqlField(index = true) private String subnetId; public PortIdSubnet() { @@ -38,4 +40,4 @@ public String getSubnetId() { public void setSubnetId(String subnetId) { this.subnetId = subnetId; } -} +} \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/dataplane/NeighborInfo.java b/web/src/main/java/com/futurewei/alcor/web/entity/dataplane/NeighborInfo.java index 40716ae7e..715e236ed 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/dataplane/NeighborInfo.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/dataplane/NeighborInfo.java @@ -17,6 +17,7 @@ free of charge, to any person obtaining a copy of this software and associated d import com.fasterxml.jackson.annotation.JsonProperty; import lombok.Data; +import org.apache.ignite.cache.query.annotations.QuerySqlField; @Data public class NeighborInfo { @@ -36,6 +37,7 @@ public class NeighborInfo { private String portIp; @JsonProperty("vpc_id") + @QuerySqlField(index = true) private String vpcId; @JsonProperty("subnet_id") @@ -142,4 +144,4 @@ public boolean equals(Object obj) { &&this.portIp.equals(o.portIp) &&this.portMac.equals(o.portMac); } -} +} \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java b/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java index ac4e39985..2157731fb 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java @@ -20,6 +20,7 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.entity.CustomerResource; import lombok.Data; import lombok.NoArgsConstructor; +import org.apache.ignite.cache.query.annotations.QuerySqlField; import java.util.List; @@ -27,6 +28,7 @@ free of charge, to any person obtaining a copy of this software and associated d @JsonInclude(JsonInclude.Include.NON_ABSENT) public class PortEntity extends CustomerResource { @JsonProperty("network_id") + @QuerySqlField(index = true) private String vpcId; @JsonProperty("admin_state_up") @@ -730,4 +732,4 @@ public Boolean getMacLearningEnabled() { public void setMacLearningEnabled(Boolean macLearningEnabled) { this.macLearningEnabled = macLearningEnabled; } -} +} \ No newline at end of file From f4e8eeffd0a2561f0ddfeee63e881ce354629595 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 8 Nov 2021 13:44:30 -0800 Subject: [PATCH 12/27] Handle only one of REST, gRPC chnnels being open in NM (TEMP) --- scripts/busybox-ping-test/helper_functions.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/busybox-ping-test/helper_functions.py b/scripts/busybox-ping-test/helper_functions.py index a6c0df037..f3ad22047 100755 --- a/scripts/busybox-ping-test/helper_functions.py +++ b/scripts/busybox-ping-test/helper_functions.py @@ -167,6 +167,9 @@ def check_alcor_services(): if retcode > 0: print("Failed to execute command", repr(str(command))) print_output(res[1]) + elif "9001 9002 9003 9004 9005 9006 9007 9008 9009 9010 9011 9012 9014 9015" in str(res): + print("SUCCESS for: ", command, "\n") + return True elif "9001 9002 9003 9004 9005 9006 9007 9008 9009 9010 9011 9012 9015 9016" in str(res): print("SUCCESS for: ", command, "\n") return True From dfa0c320e4fba1a85414d563b6f2e0c370754e83 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 8 Nov 2021 16:44:12 -0800 Subject: [PATCH 13/27] Handle name normalization of dynamic caches --- .../common/db/ignite/IgniteClientDbCache.java | 8 +++----- .../futurewei/alcor/common/utils/CommonUtil.java | 14 +++++++++++--- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 22e8501fe..ff42368f5 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -74,7 +74,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { extractSqlFields(className); if (sqlFields != null && sqlFields.size() != 0) { ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); - clientCacheConfig.setName(name); + clientCacheConfig.setName(CommonUtil.getSqlNameFromCacheName(name)); this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); } } @@ -95,7 +95,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura try { String className = v.getName(); ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); - clientCacheConfig.setName(cacheConfig.getName()); + clientCacheConfig.setName(CommonUtil.getSqlNameFromCacheName(cacheConfig.getName())); clientCacheConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); logger.log(Level.INFO, "Getting or creating cache " + clientCacheConfig.getName() + " AtomicityMode is " + clientCacheConfig.getAtomicityMode()); extractSqlFields(className); @@ -387,8 +387,7 @@ private String buildSqlFieldsQuery(Map filterParams) { * If the class has QuerySqlField annotations, add query entry fields and indexes. */ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, String className, ClientCacheConfiguration cacheConfig, ExpiryPolicy ep) { - // TODO: Rename getSchemaNameForCacheClass to something like getSqlNameFromJavaName - String cacheName = CommonUtil.getSchemaNameForCacheClass(cacheConfig.getName()); + String cacheName = CommonUtil.getSqlNameFromCacheName(cacheConfig.getName()); logger.log(Level.INFO, "Creating cache " + cacheName + " with index"); Class v; @@ -436,7 +435,6 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str } cacheConfig.setQueryEntities(qryEnt); - // String schName = CommonUtil.getSchemaNameForCacheClass(v.getName()); logger.log(Level.INFO, "Setting schema name " + SQL_SCHEMA_NAME + " for cache " + cacheName); cacheConfig.setSqlSchema(SQL_SCHEMA_NAME); diff --git a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java index 6e79f1a7b..115d2c2f7 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java +++ b/lib/src/main/java/com/futurewei/alcor/common/utils/CommonUtil.java @@ -134,7 +134,15 @@ public static String getSimpleFromCanonicalName(String canon) { * @param className * @return */ - public static String getSchemaNameForCacheClass(String className) { - return className.replaceAll("\\.", "_"); + public static String getSqlNameFromCacheName(String className) { + return className.replaceAll("[\\.-]", "_"); } -} + + /** + * @param name of a cache. Could contain dots and dashes. + * @return name with dots and dashes replaced with underscores. + */ + public static String getSqlNameFromGUID(String name) { + return name.replaceAll("\\.-", "_"); + } +} \ No newline at end of file From f89e2434d636f45045b4a86e9320dec1e9eeab75 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 8 Nov 2021 23:10:40 -0800 Subject: [PATCH 14/27] Fix filterParams being empty and fallback to scanquery --- .../alcor/common/db/ignite/IgniteClientDbCache.java | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index ff42368f5..f01243ffc 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -278,7 +278,9 @@ public Transaction getTransaction() { } private boolean checkForSqlFieldsQuery(Map filterParams) { - if (checkedForSqlFields && (sqlFields == null || sqlFields.size() == 0)) + if (!checkedForSqlFields) + return false; + if (sqlFields == null || sqlFields.size() == 0 || filterParams.size() == 0) return false; /* @@ -290,7 +292,7 @@ private boolean checkForSqlFieldsQuery(Map filterParams) { ++idxCount; } - return idxCount == filterParams.size(); + return idxCount != 0 && idxCount == filterParams.size(); } private V getBySqlFields(Map filterParams) throws CacheException { From d1d57442ec833d0ee3e4921f2de2b6cbf37ad8f8 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Tue, 9 Nov 2021 12:01:14 -0800 Subject: [PATCH 15/27] Handle failed ping test gracefully --- scripts/busybox-ping-test/container_ops.py | 2 ++ scripts/busybox-ping-test/ping_test.py | 7 +++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/scripts/busybox-ping-test/container_ops.py b/scripts/busybox-ping-test/container_ops.py index 89b07575a..62e0d78ef 100755 --- a/scripts/busybox-ping-test/container_ops.py +++ b/scripts/busybox-ping-test/container_ops.py @@ -94,6 +94,8 @@ def run_ping_test(target_machines, ip_addrs, container_names): expected_output = "2 packets transmitted, 2 packets received" if expected_output in str(output1) and expected_output in str(output2): print (colored("PING TEST SUCCESSFULL", 'green')) + return 0 else: print(colored('PING TEST FAILED', 'red')) + return 1 diff --git a/scripts/busybox-ping-test/ping_test.py b/scripts/busybox-ping-test/ping_test.py index 3b18c88d7..5f58acc8d 100755 --- a/scripts/busybox-ping-test/ping_test.py +++ b/scripts/busybox-ping-test/ping_test.py @@ -209,9 +209,12 @@ def main(): print("Goal states: ", ip_mac_db) print("Container names: ", container_names) busybox_container_deploy(aca_node_ips, ip_mac_db, container_names) - run_ping_test(aca_node_ips, goal_state_ips, container_names) + status = run_ping_test(aca_node_ips, goal_state_ips, container_names) + if status != 0: + print("ERROR: Quitting test\n") + sys.exit(1) if __name__ == "__main__": main() - + sys.exit(0) From 33c79721fd637984004b64b0bb8a3a18d06c9d43 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Fri, 12 Nov 2021 15:10:43 -0800 Subject: [PATCH 16/27] Fix name normalization --- .../common/db/ignite/IgniteClientDbCache.java | 7 +++++++ .../SqlFldScanCmp/src/META-INF/MANIFEST.MF | 3 +++ scripts/sqlfld_scan_qry_comp/pom.xml | 16 ++++++++++++++++ .../NetworkConfigManagerApplication.java | 4 ++-- 4 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF create mode 100644 scripts/sqlfld_scan_qry_comp/pom.xml diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index f01243ffc..cfe2f421a 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -22,6 +22,7 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.db.ignite.query.ScanQueryBuilder; import com.futurewei.alcor.common.logging.Logger; import com.futurewei.alcor.common.logging.LoggerFactory; +import com.futurewei.alcor.common.stats.DurationStatistics; import com.futurewei.alcor.common.utils.CommonUtil; import com.futurewei.alcor.common.utils.ControllerUtil; import org.apache.ignite.binary.BinaryObject; @@ -295,6 +296,7 @@ private boolean checkForSqlFieldsQuery(Map filterParams) { return idxCount != 0 && idxCount == filterParams.size(); } + @DurationStatistics private V getBySqlFields(Map filterParams) throws CacheException { try { Map result = runSQLFieldsQuery(filterParams); @@ -316,6 +318,7 @@ private V getBySqlFields(Map filterParams) throws Cac return null; } + @DurationStatistics public Map getBySqlFieldsAll(Map filterParams) { try { return runSQLFieldsQuery(filterParams); @@ -349,6 +352,7 @@ private Map runSQLFieldsQuery(Map filterParams) } } + @DurationStatistics private String buildSqlFieldsQuery(Map filterParams) { SqlField valFld = sqlFields.get("$value"); Class v; @@ -359,6 +363,7 @@ private String buildSqlFieldsQuery(Map filterParams) { return null; } + String valFldName = CommonUtil.getSimpleFromCanonicalName(valFld.type); StringBuilder sb = new StringBuilder("select _key, _val from " + SQL_SCHEMA_NAME + "." + cache.getConfiguration().getQueryEntities()[0].getTableName() + " where "); boolean needAnd = false; @@ -421,6 +426,8 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str } } + SqlField valFld = sqlFields.get("$value"); + qryFields.put(CommonUtil.getSimpleFromCanonicalName(valFld.type), valFld.type); qryEnt.setFields(qryFields); logger.log(Level.INFO, "QE: " + qryEnt); diff --git a/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF b/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF new file mode 100644 index 000000000..6f5dc42e5 --- /dev/null +++ b/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF @@ -0,0 +1,3 @@ +Manifest-Version: 1.0 +Main-Class: sqlfldscancmp.SqlFldScanCmp + diff --git a/scripts/sqlfld_scan_qry_comp/pom.xml b/scripts/sqlfld_scan_qry_comp/pom.xml new file mode 100644 index 000000000..a66fdf4fc --- /dev/null +++ b/scripts/sqlfld_scan_qry_comp/pom.xml @@ -0,0 +1,16 @@ + + + 4.0.0 + + SqlFldScanQryComp + SqlFldScanQryComp + 1.0-SNAPSHOT + + + 11 + 11 + + + \ No newline at end of file diff --git a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java index ff0270dbe..956de352e 100644 --- a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java +++ b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java @@ -32,13 +32,13 @@ free of charge, to any person obtaining a copy of this software and associated d @Import(TracerConfiguration.class) public class NetworkConfigManagerApplication { + /** @Autowired private NetworkConfigServer networkConfigServer; @PostConstruct public void instantiateGrpcServer(){ return; - /** * Return so that rest API endpoint can work * try { @@ -47,8 +47,8 @@ public void instantiateGrpcServer(){ } catch (Exception e) { e.printStackTrace(); } - */ } + */ public static void main(String[] args) { SpringApplication.run(NetworkConfigManagerApplication.class, args); From 74023d0bc8efe63ebd6434ae57f5300c42c8784b Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 15 Nov 2021 21:38:37 -0800 Subject: [PATCH 17/27] fix sql tables not apprearing problem --- .../alcor/common/db/ignite/IgniteClientDbCache.java | 13 +++++++++---- .../alcor/nodemanager/dao/NodeRepository.java | 6 +++++- .../src/main/resources/application.properties | 4 ++-- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index cfe2f421a..876e9e100 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -77,18 +77,20 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); clientCacheConfig.setName(CommonUtil.getSqlNameFromCacheName(name)); this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); + if (this.cache == null) { + logger.log(Level.WARNING, "Create cache for client " + className + " with index failed, falling back"); + } } } if (this.cache == null) this.cache = igniteClient.getOrCreateCache(className); - logger.log(Level.INFO, "Cache " + className + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } catch (ClientException e) { logger.log(Level.WARNING, "Create cache for client " + className + " failed:" + e.getMessage()); - logger.log(Level.WARNING, "Create cache for client " + className + " failed:" + e.getMessage()); } Assert.notNull(this.cache, "Create cache for client " + className + "failed"); + logger.log(Level.INFO, "Cache " + className + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); this.transaction = new IgniteClientTransaction(igniteClient); } @@ -102,16 +104,19 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura extractSqlFields(className); if (sqlFields != null) { this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); + if (this.cache == null) { + logger.log(Level.WARNING, "Create cache for client " + className + " with index failed, falling back"); + } } if (this.cache == null) this.cache = igniteClient.getOrCreateCache(clientCacheConfig); - logger.log(Level.INFO, "Retrieved cache " + this.cache.getConfiguration().getName() + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } catch (ClientException e) { logger.log(Level.WARNING, "Create cache for client " + cacheConfig.getName() + " failed:" + e.getMessage()); } Assert.notNull(this.cache, "Create cache for client " + cacheConfig.getName() + "failed"); + logger.log(Level.INFO, "Retrieved cache " + this.cache.getConfiguration().getName() + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); this.transaction = new IgniteClientTransaction(igniteClient); } @@ -513,4 +518,4 @@ private void extractSqlFields(String className) { // } } -} \ No newline at end of file +} diff --git a/services/node_manager/src/main/java/com/futurewei/alcor/nodemanager/dao/NodeRepository.java b/services/node_manager/src/main/java/com/futurewei/alcor/nodemanager/dao/NodeRepository.java index 9744dc5ae..bf2d7078b 100644 --- a/services/node_manager/src/main/java/com/futurewei/alcor/nodemanager/dao/NodeRepository.java +++ b/services/node_manager/src/main/java/com/futurewei/alcor/nodemanager/dao/NodeRepository.java @@ -21,7 +21,10 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.db.Transaction; import com.futurewei.alcor.common.db.repo.ICacheRepository; import com.futurewei.alcor.common.stats.DurationStatistics; +import com.futurewei.alcor.common.utils.CommonUtil; import com.futurewei.alcor.web.entity.node.NodeInfo; +import org.apache.ignite.cache.CacheAtomicityMode; +import org.apache.ignite.configuration.CacheConfiguration; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; @@ -40,7 +43,8 @@ public class NodeRepository implements ICacheRepository { @Autowired public NodeRepository(CacheFactory cacheFactory) { - cache = cacheFactory.getCache(NodeInfo.class, "nmm_nodeinfo_cache"); + CacheConfiguration cacheConfig = CommonUtil.getCacheConfiguration("nmm_nodeinfo_cache"); + cache = cacheFactory.getCache(NodeInfo.class, cacheConfig); } public ICache getCache() { diff --git a/services/node_manager/src/main/resources/application.properties b/services/node_manager/src/main/resources/application.properties index 4128b2a8b..17b8084ad 100644 --- a/services/node_manager/src/main/resources/application.properties +++ b/services/node_manager/src/main/resources/application.properties @@ -35,7 +35,7 @@ ignite.thin.client.enable=true #####Logging configuration##### #logging.file.path=./ #logging.file.name=node-manager.log -#logging.level.root=INFO +logging.level.root=DEBUG ## Misc: MULTIPART (MultipartProperties) # Enable multipart uploads @@ -58,4 +58,4 @@ opentracing.jaeger.enabled=true opentracing.jaeger.log-spans=true opentracing.jaeger.enable-w3c-propagation=true opentracing.jaeger.enable-b3-propagation=true -opentracing.jaeger.service-name=alcor-node \ No newline at end of file +opentracing.jaeger.service-name=alcor-node From 557a35f6dc0a0bc0038324f5416095b07dadc1b1 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Tue, 16 Nov 2021 17:31:33 -0800 Subject: [PATCH 18/27] Make SQL Query benchmark a service to avoid insert and query operating on different objects --- services/sqlquery/pom.xml | 67 ++++ .../futurewei/alcor/sqlquery/sqlquery.java | 297 ++++++++++++++++++ 2 files changed, 364 insertions(+) create mode 100644 services/sqlquery/pom.xml create mode 100644 services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java diff --git a/services/sqlquery/pom.xml b/services/sqlquery/pom.xml new file mode 100644 index 000000000..554e21276 --- /dev/null +++ b/services/sqlquery/pom.xml @@ -0,0 +1,67 @@ + + + 4.0.0 + + com.futurewei.alcor + sqlquery + 1.0-SNAPSHOT + + + 11 + 11 + + + + + com.futurewei.alcor + common + 0.1.0-SNAPSHOT + compile + + + com.futurewei.alcor + web + 0.1.0-SNAPSHOT + compile + + + org.apache.ignite + ignite-core + 2.10.0 + compile + + + org.apache.ignite + ignite-indexing + 2.10.0 + compile + + + org.apache.ignite + ignite-spring + 2.10.0 + compile + + + + + + + org.apache.maven.plugins + maven-jar-plugin + 3.0.0 + + + + true + lib/ + com.futurewei.alcor.sqlquery.SqlQuery2 + + + + + + + \ No newline at end of file diff --git a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java new file mode 100644 index 000000000..3a0eda7ff --- /dev/null +++ b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java @@ -0,0 +1,297 @@ +package com.futurewei.alcor.sqlquery; +import org.apache.ignite.Ignition; +import org.apache.ignite.cache.CacheAtomicityMode; +import org.apache.ignite.cache.CacheKeyConfiguration; +import org.apache.ignite.cache.QueryEntity; +import org.apache.ignite.cache.QueryIndex; +import org.apache.ignite.cache.query.QueryCursor; +import org.apache.ignite.cache.query.SqlFieldsQuery; +import org.apache.ignite.client.ClientCache; +import org.apache.ignite.client.ClientCacheConfiguration; +import org.apache.ignite.client.ClientTransaction; +import org.apache.ignite.client.IgniteClient; +import org.apache.ignite.configuration.ClientConfiguration; +import org.apache.ignite.transactions.Transaction; +import org.json.simple.*; +import org.json.simple.JSONObject; +import org.json.simple.parser.JSONParser; +import org.w3c.dom.Node; +import com.futurewei.alcor.web.entity.node.NodeInfo; + +import java.io.BufferedReader; +import java.io.BufferedWriter; +import java.io.FileReader; +import java.io.FileWriter; +import java.util.*; + +public class sqlquery { + static int[] ipAddress = new int[4]; + static int[] macAddress = new int[6]; + static String igniteAddress; + static String nodePrefix = "node"; + static String ncmPrefix = "ncm"; + static int ncmNum = 0; + static int ncmMax = 1000; + static boolean generateData = false; + + public sqlquery(String uuid) { + } + + public static void main(String[] args) { + + if (args.length < 3) { + System.out.println("need input: json_output_filename number_of_entries number_of_queries [-g(enerate data)]"); + System.exit(-1); + } + final String tblName = "nmm_nodeinfo_cache"; + final String schName = "alcor"; + // final String nodeInfoValueType = "sqlfldscancmp.NodeInfo"; + + int exitCode = -1; + + String jsonFile = args[0]; + int entryCount = Integer.valueOf(args[1]); + int qryCount = Integer.valueOf(args[2]); + if (args.length > 3 && args[3].equals("-g")) + generateData = true; + + if (args.length > 4) + igniteAddress = args[4]; + else + igniteAddress = "127.0.0.1"; + + ipAddress[0] = ipAddress[1] = ipAddress[2] = ipAddress[3] = 1; + macAddress[0] = macAddress[1] = macAddress[2] = macAddress[3] = macAddress[4] = macAddress[5] = 1; + ClientConfiguration clientCfg = new ClientConfiguration(); + clientCfg.setPartitionAwarenessEnabled(true); + clientCfg.setAddresses(igniteAddress + ":10800"); + + System.out.println("ARGUMENTS"); + System.out.println("jsonFile = " + jsonFile); + System.out.println("entryCount = " + entryCount); + System.out.println("qryCount = " + qryCount); + System.out.println("generateData = " + generateData); + System.out.println("igniteAddress = " + igniteAddress); + IgniteClient client = null; + + try { + client = Ignition.startClient(clientCfg); + } + catch (Exception e) { + System.out.println("Cannot connect to Local server: " + e.getMessage()); + System.exit(exitCode); + } + + System.out.println("Testing QueryEntity, Indexing, SQL Access to ClientCache from ThinClient"); + + try { + + // JSONObject jsonObject = (JSONObject)(new JSONParser().parse(new FileReader(jsonFile))); + // JSONArray jsonArray = (JSONArray)jsonObject.get("host_infos"); + ClientCacheConfiguration nodeInfoCacheConfig = new ClientCacheConfiguration(); + nodeInfoCacheConfig.setName(tblName); + nodeInfoCacheConfig.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); + CacheKeyConfiguration keyConf = new CacheKeyConfiguration(); + + QueryEntity qryEnt = new QueryEntity(); + System.out.println("NodeInfo type = " + NodeInfo.class.getName()); + qryEnt.setValueType(NodeInfo.class.getName()); + LinkedHashMap qryFields = new LinkedHashMap<>(); + + qryFields.put("id", String.class.getName()); + qryFields.put("name", String.class.getName()); + + qryEnt.setFields(qryFields); + + qryEnt.setIndexes(Arrays.asList( + new QueryIndex("id"), + new QueryIndex("name"))); + nodeInfoCacheConfig.setQueryEntities(qryEnt).setSqlSchema(schName); + + ClientCache nodeInfoClientCache = client.getOrCreateCache(nodeInfoCacheConfig); + + int ncmSequene = 0; + int batchCount = 0; + int qrySequence = 0; + long insBegin = 0, insEnd = 0; + + String[] queryNames = new String[qryCount]; + if (generateData) { + // insert data, commit every 10000 entries + int commitSize = 10000; + BufferedWriter outFile = new BufferedWriter(new FileWriter(jsonFile)); + ClientTransaction tx = null; + boolean inTxn = false; + try { + insBegin = System.nanoTime(); + for (int e = 0; e < entryCount; ++e) { + NodeInfo nodeInfo = createNodeInfo(nodePrefix, ncmPrefix, e, ncmNum); + if (!inTxn) { + tx = client.transactions().txStart(); + inTxn = true; + } + nodeInfoClientCache.put(nodeInfo.getId(), nodeInfo); + if (batchCount++ >= commitSize) { + tx.commit(); + inTxn = false; + batchCount = 0; + } + + if (ncmSequene++ >= ncmMax) + ncmSequene = 0; + + outFile.write(nodeInfo.getId() + "|" + nodeInfo.getName() + "|" + + nodeInfo.getNcmId() + "|" + nodeInfo.getLocalIp() + "|" + + nodeInfo.getMacAddress() + "\n"); + if (qrySequence < qryCount) + queryNames[qrySequence++] = nodeInfo.getName(); + } + insEnd = System.nanoTime(); + if (inTxn) + tx.commit(); + outFile.close(); + } + catch (Exception e) { + System.out.println("Failed to insert: " + e.getMessage()); + System.exit(-1); + } + } + else { + BufferedReader infile = new BufferedReader(new FileReader(jsonFile)); + while (qrySequence < qryCount) { + String line = infile.readLine(); + String[] fields = line.split("|"); + queryNames[qrySequence++] = new String(fields[1]); + } + infile.close(); + } + + System.out.println("DONE INSERTING"); + + SqlFieldsQuery sql = new SqlFieldsQuery("select _key, name, cast(_val as varchar(128)) from " + schName + + "." + tblName + " where name = ?"); + + int i; + long[] qryTime = new long[qryCount]; + long[] curTime = new long[qryCount]; + int recCount = 0; + try { + + long qbegin, qend, cbeign, cend = 0; + for (i = 0; i < qryCount; ++i) { + String nodeNameIn = queryNames[i]; + sql.setArgs(nodeNameIn); + qbegin = System.nanoTime(); + QueryCursor> cursor = nodeInfoClientCache.query(sql); + qend = System.nanoTime(); + qryTime[i] = (qend - qbegin) / 1000; + String nodeId = null, nodeName = null; + cbeign = qend; + try { + for (List row : cursor) { + ++recCount; + cbeign = System.nanoTime(); + nodeId = row.get(0).toString(); + nodeName = row.get(1).toString(); + cend = System.nanoTime(); + // assert(nodeNameIn.equals(nodeName)); + curTime[i] = (cend - cbeign) / 1000; + } + } + catch (Exception e) { + System.out.println("Cursor failed: " + e.getMessage()); + continue; + } + + System.out.println("SQL: " + sql + " args: " + sql.getArgs().toString()); + exitCode = 0; + } + + System.out.println("INSERT_TIME " + (insEnd - insBegin) / 1000 + " us"); + System.out.println("REC_COUNT = " + recCount); + + for (int j = 0; j < i; ++j) { + System.out.println(qryTime[j] + "\t" + curTime[j]); + } + } catch (Exception e) { + System.out.println("SQL Query failed: " + e.getMessage()); + } + } catch (Exception e) { + System.out.println("Failed to instantiate PersonCache : " + e.getMessage()); + } finally { + try { + if (client != null) + client.close(); + } catch (Exception e) { + } + System.exit(exitCode); + } + } + + static NodeInfo createNodeInfo(String nodePrefix, String ncmPrefix, int nodeNumber, int ncmNumber) + { + String nodeId = UUID.randomUUID().toString(); + String nodeName = nodePrefix + "_" + String.format("%07d", nodeNumber); + String nodeIp = getNextIp(); + String nodeMac = getNextMAC(); + String ncmId = ncmPrefix + "_" + String.format("%03d", ncmNumber); + + NodeInfo newNode = new NodeInfo(nodeId, nodeName, nodeIp, nodeMac); + newNode.setNcmId(ncmId); + + return newNode; + } + + static String getNextIp() { + if (ipAddress[0]++ > 254) { + ipAddress[0] = 0; + if (ipAddress[1]++ > 254) { + ipAddress[1] = 0; + if (ipAddress[2]++ > 254) { + ipAddress[2] = 0; + if (ipAddress[3]++ > 254) { + System.out.println("IP Address out of bounds"); + System.exit(-1); + } + } + } + } + + String newIp = String.format("%d", ipAddress[3]) + "." + + String.format("%d", ipAddress[2]) + "." + + String.format("%d", ipAddress[1]) + "." + + String.format("%d", ipAddress[0]); + + return newIp; + } + + static String getNextMAC() + { + if (macAddress[0]++ > 255) { + macAddress[0] = 0; + if (macAddress[1]++ > 255) { + macAddress[1] = 0; + if (macAddress[2]++ > 255) { + macAddress[2] = 0; + if (macAddress[3]++ > 255) { + macAddress[3] = 0; + if (macAddress[4]++ > 255) { + if (macAddress[5]++ > 255) { + System.out.println("MAC out of range"); + System.exit(-1); + } + } + } + } + } + } + String newMac = String.format("%02x", macAddress[5]) + ":" + + String.format("%02x", macAddress[4]) + ":" + + String.format("%02x", macAddress[3]) + ":" + + String.format("%02x", macAddress[2]) + ":" + + String.format("%02x", macAddress[1]) + ":" + + String.format("%02x", macAddress[0]); + + return newMac; + } +} \ No newline at end of file From 67fe49263c7231e4ff706aaa590a36eedef6c354 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 17 Nov 2021 09:38:22 -0800 Subject: [PATCH 19/27] Query _VAL field instead of specific filed, this works if run with mvn exec --- scripts/sqlfld_scan_qry_comp/pom.xml | 16 ---------------- .../com/futurewei/alcor/sqlquery/sqlquery.java | 7 ++++--- 2 files changed, 4 insertions(+), 19 deletions(-) delete mode 100644 scripts/sqlfld_scan_qry_comp/pom.xml diff --git a/scripts/sqlfld_scan_qry_comp/pom.xml b/scripts/sqlfld_scan_qry_comp/pom.xml deleted file mode 100644 index a66fdf4fc..000000000 --- a/scripts/sqlfld_scan_qry_comp/pom.xml +++ /dev/null @@ -1,16 +0,0 @@ - - - 4.0.0 - - SqlFldScanQryComp - SqlFldScanQryComp - 1.0-SNAPSHOT - - - 11 - 11 - - - \ No newline at end of file diff --git a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java index 3a0eda7ff..4e08c5b25 100644 --- a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java +++ b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java @@ -168,7 +168,7 @@ public static void main(String[] args) { System.out.println("DONE INSERTING"); - SqlFieldsQuery sql = new SqlFieldsQuery("select _key, name, cast(_val as varchar(128)) from " + schName + + SqlFieldsQuery sql = new SqlFieldsQuery("select _key, _val, cast(_val as varchar(128)) from " + schName + "." + tblName + " where name = ?"); int i; @@ -192,9 +192,10 @@ public static void main(String[] args) { ++recCount; cbeign = System.nanoTime(); nodeId = row.get(0).toString(); - nodeName = row.get(1).toString(); + NodeInfo node = (NodeInfo)row.get(1); + nodeName = node.getName(); // row.get(1).toString(); cend = System.nanoTime(); - // assert(nodeNameIn.equals(nodeName)); + assert(nodeNameIn.equals(nodeName)); curTime[i] = (cend - cbeign) / 1000; } } From 181f89834b90dd9e04367a29a7d1677237e65a73 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 17 Nov 2021 15:00:23 -0800 Subject: [PATCH 20/27] Add Scan query benchmark --- services/scanquery/pom.xml | 61 ++++ .../futurewei/alcor/scanquery/scanquery.java | 280 ++++++++++++++++++ .../alcor/web/entity/node/NodeInfo.java | 4 +- 3 files changed, 343 insertions(+), 2 deletions(-) create mode 100644 services/scanquery/pom.xml create mode 100644 services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java diff --git a/services/scanquery/pom.xml b/services/scanquery/pom.xml new file mode 100644 index 000000000..061595497 --- /dev/null +++ b/services/scanquery/pom.xml @@ -0,0 +1,61 @@ + + + 4.0.0 + + com.futurewei.alcor + scanquery + 1.0-SNAPSHOT + + + 11 + 11 + + + + + com.futurewei.alcor + common + 0.1.0-SNAPSHOT + compile + + + com.futurewei.alcor + web + 0.1.0-SNAPSHOT + compile + + + org.apache.ignite + ignite-core + 2.10.0 + compile + + + org.apache.ignite + ignite-spring + 2.10.0 + compile + + + + + + + org.apache.maven.plugins + maven-jar-plugin + 3.0.0 + + + + true + lib/ + com.futurewei.alcor.scanquery.scanquery + + + + + + + \ No newline at end of file diff --git a/services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java b/services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java new file mode 100644 index 000000000..a477cc710 --- /dev/null +++ b/services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java @@ -0,0 +1,280 @@ +package com.futurewei.alcor.scanquery; +import com.futurewei.alcor.common.db.ignite.query.MapPredicate; +import com.futurewei.alcor.common.db.ignite.query.ScanQueryBuilder; +import org.apache.ignite.Ignition; +import org.apache.ignite.cache.CacheAtomicityMode; +import org.apache.ignite.cache.CacheKeyConfiguration; +import org.apache.ignite.cache.query.QueryCursor; +import org.apache.ignite.client.ClientCache; +import org.apache.ignite.client.ClientCacheConfiguration; +import org.apache.ignite.client.ClientTransaction; +import org.apache.ignite.client.IgniteClient; +import org.apache.ignite.configuration.ClientConfiguration; +import org.apache.ignite.lang.IgniteBiPredicate; +import org.apache.ignite.transactions.Transaction; +import org.apache.ignite.binary.BinaryObject; +import org.apache.ignite.lang.IgniteClosure; +import com.futurewei.alcor.web.entity.node.NodeInfo; + +import java.io.BufferedReader; +import java.io.BufferedWriter; +import java.io.FileReader; +import java.io.FileWriter; +import javax.cache.Cache; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.UUID; +import java.util.concurrent.locks.Lock; +import java.util.function.BinaryOperator; + +public class scanquery { + static int[] ipAddress = new int[4]; + static int[] macAddress = new int[6]; + static String igniteAddress; + static String nodePrefix = "node"; + static String ncmPrefix = "ncm"; + static int ncmNum = 0; + static int ncmMax = 1000; + static boolean generateData = false; + + public scanquery(String uuid) { + } + + public static void main(String[] args) { + + if (args.length < 3) { + System.out.println("need input: json_output_filename number_of_entries number_of_queries [-g(enerate data)] [ignite ipaddress]"); + System.exit(-1); + } + final String tblName = "nmm_nodeinfo_cache"; + final String schName = "alcor"; + + int exitCode = -1; + + String jsonFile = args[0]; + int entryCount = Integer.valueOf(args[1]); + int qryCount = Integer.valueOf(args[2]); + if (args.length > 3 && args[3].equals("-g")) + generateData = true; + + if (args.length > 4) + igniteAddress = args[4]; + else + igniteAddress = "127.0.0.1"; + + ipAddress[0] = ipAddress[1] = ipAddress[2] = ipAddress[3] = 1; + macAddress[0] = macAddress[1] = macAddress[2] = macAddress[3] = macAddress[4] = macAddress[5] = 1; + ClientConfiguration clientCfg = new ClientConfiguration(); + clientCfg.setPartitionAwarenessEnabled(true); + clientCfg.setAddresses(igniteAddress + ":10800"); + + System.out.println("ARGUMENTS"); + System.out.println("jsonFile = " + jsonFile); + System.out.println("entryCount = " + entryCount); + System.out.println("qryCount = " + qryCount); + System.out.println("generateData = " + generateData); + System.out.println("igniteAddress = " + igniteAddress); + IgniteClient client = null; + + try { + client = Ignition.startClient(clientCfg); + } + catch (Exception e) { + System.out.println("Cannot connect to server at : " + + igniteAddress + ", Error: " + e.getMessage()); + System.exit(exitCode); + } + + System.out.println("Testing Scanquery from ThinClient"); + + try { + + ClientCacheConfiguration nodeInfoCacheConfig = new ClientCacheConfiguration(); + nodeInfoCacheConfig.setName(tblName); + nodeInfoCacheConfig.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); + ClientCache nodeInfoClientCache = client.getOrCreateCache(nodeInfoCacheConfig); + + int ncmSequene = 0; + int batchCount = 0; + int qrySequence = 0; + long insBegin = 0, insEnd = 0; + + String[] queryNames = new String[qryCount]; + if (generateData) { + // insert data, commit every 10000 entries + int commitSize = 10000; + BufferedWriter outFile = new BufferedWriter(new FileWriter(jsonFile)); + ClientTransaction tx = null; + boolean inTxn = false; + try { + insBegin = System.nanoTime(); + for (int e = 0; e < entryCount; ++e) { + NodeInfo nodeInfo = createNodeInfo(nodePrefix, ncmPrefix, e, ncmNum); + if (!inTxn) { + tx = client.transactions().txStart(); + inTxn = true; + } + nodeInfoClientCache.put(nodeInfo.getId(), nodeInfo); + if (batchCount++ >= commitSize) { + tx.commit(); + inTxn = false; + batchCount = 0; + } + + if (ncmSequene++ >= ncmMax) + ncmSequene = 0; + + outFile.write(nodeInfo.getId() + "|" + nodeInfo.getName() + "|" + + nodeInfo.getNcmId() + "|" + nodeInfo.getLocalIp() + "|" + + nodeInfo.getMacAddress() + "\n"); + if (qrySequence < qryCount) + queryNames[qrySequence++] = nodeInfo.getName(); + } + insEnd = System.nanoTime(); + if (inTxn) + tx.commit(); + outFile.close(); + } + catch (Exception e) { + System.out.println("Failed to insert: " + e.getMessage()); + System.exit(-1); + } + } + else { + BufferedReader infile = new BufferedReader(new FileReader(jsonFile)); + while (qrySequence < qryCount) { + String line = infile.readLine(); + String[] fields = line.split("|"); + queryNames[qrySequence++] = new String(fields[1]); + } + infile.close(); + } + + System.out.println("DONE INSERTING"); + + System.out.println("Running ScanQuery, Alcor version"); + + long[] qryTime = new long[qryCount]; + long[] curTime = new long[qryCount]; + int recCount = 0; + try { + long qbegin, qend, cbegin, cend = 0; + for (int i = 0; i < qryCount; ++i) { + String nodeNameIn = queryNames[i]; + qbegin = System.nanoTime(); + Map queryParams = new HashMap<>(); + Object[] values = new Object[1]; + values[0] = nodeNameIn; + queryParams.put("name", values); + IgniteBiPredicate pred = MapPredicate.getInstance(queryParams); + QueryCursor> cursor = nodeInfoClientCache.withKeepBinary().query( + ScanQueryBuilder.newScanQuery(pred)); + try { + qbegin = System.nanoTime(); + List> result = cursor.getAll(); + qend = System.nanoTime(); + qryTime[i] = (qend - qbegin) / 1000; + if (result.isEmpty()) + continue; + cbegin = System.nanoTime(); + BinaryObject obj = result.get(0).getValue(); + cend = System.nanoTime(); + curTime[i] = (cend - cbegin) / 1000; + if (obj instanceof BinaryObject) { + ++recCount; + BinaryObject binObj = (BinaryObject) obj; + NodeInfo node = (NodeInfo) binObj.deserialize(); + assert(node.getName().equals(nodeNameIn)); + } + } + catch (Exception e) { + System.out.println("Scan Query cursor failed " + e.getMessage()); + break; + } + } + exitCode = 0; + } + catch (Exception e) { + System.out.println("Scan Query instantiaon failed: " + e.getMessage()); + } + + System.out.println("INSERT_TIME " + (insEnd - insBegin) / 1000 + " us"); + System.out.println("REC_COUNT = " + recCount); + + for (int i = 0; i < qryCount; ++i) { + System.out.println(qryTime[i] + "\t" + curTime[i]); + } + } catch (Exception e) { + System.out.println("Failed to instantiate PersonCache : " + e.getMessage()); + } + System.exit(exitCode); + } + + static NodeInfo createNodeInfo(String nodePrefix, String ncmPrefix, int nodeNumber, int ncmNumber) + { + String nodeId = UUID.randomUUID().toString(); + String nodeName = nodePrefix + "_" + String.format("%07d", nodeNumber); + String nodeIp = getNextIp(); + String nodeMac = getNextMAC(); + String ncmId = ncmPrefix + "_" + String.format("%03d", ncmNumber); + + NodeInfo newNode = new NodeInfo(nodeId, nodeName, nodeIp, nodeMac); + newNode.setNcmId(ncmId); + + return newNode; + } + + static String getNextIp() { + if (ipAddress[0]++ > 254) { + ipAddress[0] = 0; + if (ipAddress[1]++ > 254) { + ipAddress[1] = 0; + if (ipAddress[2]++ > 254) { + ipAddress[2] = 0; + if (ipAddress[3]++ > 254) { + System.out.println("IP Address out of bounds"); + System.exit(-1); + } + } + } + } + + String newIp = String.format("%d", ipAddress[3]) + "." + + String.format("%d", ipAddress[2]) + "." + + String.format("%d", ipAddress[1]) + "." + + String.format("%d", ipAddress[0]); + + return newIp; + } + + static String getNextMAC() + { + if (macAddress[0]++ > 255) { + macAddress[0] = 0; + if (macAddress[1]++ > 255) { + macAddress[1] = 0; + if (macAddress[2]++ > 255) { + macAddress[2] = 0; + if (macAddress[3]++ > 255) { + macAddress[3] = 0; + if (macAddress[4]++ > 255) { + if (macAddress[5]++ > 255) { + System.out.println("MAC out of range"); + System.exit(-1); + } + } + } + } + } + } + String newMac = String.format("%02x", macAddress[5]) + ":" + + String.format("%02x", macAddress[4]) + ":" + + String.format("%02x", macAddress[3]) + ":" + + String.format("%02x", macAddress[2]) + ":" + + String.format("%02x", macAddress[1]) + ":" + + String.format("%02x", macAddress[0]); + + return newMac; + } +} \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java index 82643f003..a9d04d7b8 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java @@ -33,11 +33,11 @@ public class NodeInfo implements Serializable { private static final Logger logger = LoggerFactory.getLogger(NodeInfo.class); @JsonProperty("node_id") - @QuerySqlField(index = true) + // @QuerySqlField(index = true) private String id; @JsonProperty("node_name") - @QuerySqlField(index = true) + // @QuerySqlField(index = true) private String name; @JsonProperty("local_ip") From 78ad65068e02d844ad4589f8d858e0be947cb4d8 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Fri, 19 Nov 2021 15:01:05 -0800 Subject: [PATCH 21/27] Ignite SqlFieldsQuery changes --- .../common/db/ignite/IgniteClientDbCache.java | 72 ++++++------------- .../alcor/common/entity/CustomerResource.java | 4 +- .../SqlFldScanCmp/src/META-INF/MANIFEST.MF | 3 - .../futurewei/alcor/sqlquery/sqlquery.java | 10 +-- .../alcor/web/entity/node/NodeInfo.java | 2 +- 5 files changed, 27 insertions(+), 64 deletions(-) delete mode 100644 scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 876e9e100..8cdc07fdf 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -54,7 +54,7 @@ free of charge, to any person obtaining a copy of this software and associated d public class IgniteClientDbCache implements IgniteICache { private static final Logger logger = LoggerFactory.getLogger(); - private static final String NON_SCALAR_ROWSET = "more than one rows found!"; + private static final String NON_SCALAR_ROWSET = "too many rows found!"; private static final int RESULT_THRESHOLD_SIZE = 100000; private static final String SQL_SCHEMA_NAME = "alcor"; private ClientCache cache; @@ -64,7 +64,8 @@ private static class SqlField { public boolean isIndexed; } private Map sqlFields = null; // needed for index creation and querying - private List sqlColumns = null; // needed for constructing select list + private List sqlColumns = null; // needed for constructing indexes in specified order + // so that a prefix query can use indexed search. private boolean checkedForSqlFields = false; public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { @@ -86,10 +87,10 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { this.cache = igniteClient.getOrCreateCache(className); } catch (ClientException e) { - logger.log(Level.WARNING, "Create cache for client " + className + " failed:" + e.getMessage()); + logger.log(Level.WARNING, "Create cache for client " + className + " failed: " + e.getMessage()); } - Assert.notNull(this.cache, "Create cache for client " + className + "failed"); + Assert.notNull(this.cache, "Create cache for client " + className + " failed"); logger.log(Level.INFO, "Cache " + className + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); this.transaction = new IgniteClientTransaction(igniteClient); } @@ -102,7 +103,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura clientCacheConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); logger.log(Level.INFO, "Getting or creating cache " + clientCacheConfig.getName() + " AtomicityMode is " + clientCacheConfig.getAtomicityMode()); extractSqlFields(className); - if (sqlFields != null) { + if (sqlFields != null && sqlFields.size() != 0) { this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); if (this.cache == null) { logger.log(Level.WARNING, "Create cache for client " + className + " with index failed, falling back"); @@ -112,10 +113,10 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura this.cache = igniteClient.getOrCreateCache(clientCacheConfig); } catch (ClientException e) { - logger.log(Level.WARNING, "Create cache for client " + cacheConfig.getName() + " failed:" + e.getMessage()); + logger.log(Level.WARNING, "Create cache for client " + cacheConfig.getName() + " failed: " + e.getMessage()); } - Assert.notNull(this.cache, "Create cache for client " + cacheConfig.getName() + "failed"); + Assert.notNull(this.cache, "Create cache for client " + cacheConfig.getName() + " failed"); logger.log(Level.INFO, "Retrieved cache " + this.cache.getConfiguration().getName() + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); this.transaction = new IgniteClientTransaction(igniteClient); } @@ -125,7 +126,7 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name, E if (!checkedForSqlFields) { checkedForSqlFields = true; extractSqlFields(v.getName()); - if (sqlFields != null) { + if (sqlFields != null && sqlFields.size() != 0) { ClientCacheConfiguration clientCacheConfig = new ClientCacheConfiguration(); clientCacheConfig.setName(name); getOrCreateIndexedCache(igniteClient, v.getName(), clientCacheConfig, ep); @@ -136,10 +137,10 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name, E logger.log(Level.INFO, "Cache " + name + " AtomicityMode is " + this.cache.getConfiguration().getAtomicityMode()); } } catch (ClientException e) { - logger.log(Level.WARNING, "Create cache for client " + name + " failed:" + e.getMessage()); + logger.log(Level.WARNING, "Create cache for client " + name + " failed: " + e.getMessage()); } - Assert.notNull(this.cache, "Create cache for client " + name + "failed"); + Assert.notNull(this.cache, "Create cache for client " + name + " failed"); this.transaction = new IgniteClientTransaction(igniteClient); } @@ -149,7 +150,7 @@ public V get(K key) throws CacheException { try { return cache.get(key); } catch (ClientException e) { - logger.log(Level.WARNING, "IgniteCache get operation error:" + e.getMessage()); + logger.log(Level.WARNING, "IgniteCache get operation error: " + e.getMessage()); throw new CacheException(e.getMessage()); } } @@ -159,7 +160,7 @@ public void put(K key, V value) throws CacheException { try { cache.put(key, value); } catch (ClientException e) { - logger.log(Level.WARNING, "IgniteCache put operation error:" + e.getMessage()); + logger.log(Level.WARNING, "IgniteCache put operation error: " + e.getMessage()); throw new CacheException(e.getMessage()); } } @@ -174,7 +175,7 @@ public boolean containsKey(K key) throws CacheException { try { return cache.containsKey(key); } catch (ClientException e) { - logger.log(Level.WARNING, "IgniteCache containsKey operation error:" + e.getMessage()); + logger.log(Level.WARNING, "IgniteCache containsKey operation error: " + e.getMessage()); throw new CacheException(e.getMessage()); } } @@ -197,7 +198,7 @@ public void putAll(Map items) throws CacheException { try { cache.putAll(items); } catch (ClientException e) { - logger.log(Level.WARNING, "IgniteCache putAll operation error:" + e.getMessage()); + logger.log(Level.WARNING, "IgniteCache putAll operation error: " + e.getMessage()); throw new CacheException(e.getMessage()); } } @@ -207,7 +208,7 @@ public boolean remove(K key) throws CacheException { try { return cache.remove(key); } catch (ClientException e) { - logger.log(Level.WARNING, "IgniteCache remove operation error:" + e.getMessage()); + logger.log(Level.WARNING, "IgniteCache remove operation error: " + e.getMessage()); throw new CacheException(e.getMessage()); } } @@ -239,7 +240,7 @@ public V get(IgniteBiPredicate igniteBiPredicate) throws CacheE BinaryObject binaryObject = (BinaryObject)obj; return binaryObject.deserialize(); }else{ - throw new CacheException("no support for object type:" + obj.getClass().getName()); + throw new CacheException("no support for object type: " + obj.getClass().getName()); } } @@ -258,7 +259,7 @@ public Map getAll(IgniteBiPredicate igniteBiPredicate) th cache.withKeepBinary().query(ScanQueryBuilder.newScanQuery(igniteBiPredicate)); List> result = cursor.getAll(); if(result.size() >= RESULT_THRESHOLD_SIZE){ - throw new CacheException("too many rows found!"); + throw new CacheException(NON_SCALAR_ROWSET); } Map values = new HashMap<>(result.size()); for(Cache.Entry entry: result){ @@ -267,7 +268,7 @@ public Map getAll(IgniteBiPredicate igniteBiPredicate) th BinaryObject binaryObject = (BinaryObject)obj; values.put((K)entry.getKey(), binaryObject.deserialize()); }else{ - throw new CacheException("no support for object type:" + obj.getClass().getName()); + throw new CacheException("no support for object type: " + obj.getClass().getName()); } } return values; @@ -359,16 +360,6 @@ private Map runSQLFieldsQuery(Map filterParams) @DurationStatistics private String buildSqlFieldsQuery(Map filterParams) { - SqlField valFld = sqlFields.get("$value"); - Class v; - try { - v = Class.forName(valFld.type); - } catch (Exception e) { - logger.log(Level.INFO, "Failed to find type of " + valFld.type); - return null; - } - - String valFldName = CommonUtil.getSimpleFromCanonicalName(valFld.type); StringBuilder sb = new StringBuilder("select _key, _val from " + SQL_SCHEMA_NAME + "." + cache.getConfiguration().getQueryEntities()[0].getTableName() + " where "); boolean needAnd = false; @@ -417,11 +408,6 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str ArrayList idxFields = new ArrayList<>(); try { for (String c : sqlColumns) { - if (c.equals("$value")) - continue; - // qryEnt.setKeyFieldName(idxFld.name); - // qryEnt.setKeyType(idxFld.name); - // qryEnt.setValueType(className); SqlField f = sqlFields.get(c); logger.log(Level.INFO, "getOrCreateIndexedCache: Adding " + c + " to query fields " + " in " + cacheName); qryFields.put(c, f.type); @@ -431,17 +417,8 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str } } - SqlField valFld = sqlFields.get("$value"); - qryFields.put(CommonUtil.getSimpleFromCanonicalName(valFld.type), valFld.type); qryEnt.setFields(qryFields); - - logger.log(Level.INFO, "QE: " + qryEnt); - - // qryEnt.setKeyType(valFld.name); - // qryEnt.setValueType(valFld.type); - qryEnt.setIndexes(idxFields); - logger.log(Level.INFO, "cache : " + cacheName + ", QE = " + qryEnt.toString()); } catch (Exception e) { logger.log(Level.WARNING, "Failed to create index on cache: " + cacheName + ": " + e.getMessage()); @@ -504,18 +481,11 @@ private void extractSqlFields(String className) { } } - if (!(localFields.isEmpty() || localFields.containsValue(className))) { - SqlField sqlField = new SqlField(); - sqlField.isIndexed = false; - sqlField.type = v.getTypeName(); - logger.log(Level.INFO, "QuerySqlField Adding value for _VAL with type " + sqlField.type); - localFields.put("$value", sqlField); - } sqlFields = localFields; sqlColumns = localColumns; logger.log(Level.INFO, "QuerySqlField Found " + sqlFields.size() + " sqlFields"); } catch (Exception e) { - // + logger.log(Level.INFO, "Failed to get Class info for class " + className + ", or declared fields"); } } -} +} \ No newline at end of file diff --git a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java index 98555e79f..980bcd9b9 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java +++ b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java @@ -24,7 +24,7 @@ free of charge, to any person obtaining a copy of this software and associated d public class CustomerResource extends Resource { @JsonProperty("project_id") - @QuerySqlField(index = true) + // @QuerySqlField(index = true) private String projectId; @JsonProperty("tenant_id") @@ -112,4 +112,4 @@ public String toString() { ", description='" + description + '\'' + '}'; } -} \ No newline at end of file +} diff --git a/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF b/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF deleted file mode 100644 index 6f5dc42e5..000000000 --- a/scripts/scan_sql_comp/SqlFldScanCmp/src/META-INF/MANIFEST.MF +++ /dev/null @@ -1,3 +0,0 @@ -Manifest-Version: 1.0 -Main-Class: sqlfldscancmp.SqlFldScanCmp - diff --git a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java index 4e08c5b25..68c9a5377 100644 --- a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java +++ b/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java @@ -45,7 +45,6 @@ public static void main(String[] args) { } final String tblName = "nmm_nodeinfo_cache"; final String schName = "alcor"; - // final String nodeInfoValueType = "sqlfldscancmp.NodeInfo"; int exitCode = -1; @@ -86,15 +85,12 @@ public static void main(String[] args) { try { - // JSONObject jsonObject = (JSONObject)(new JSONParser().parse(new FileReader(jsonFile))); - // JSONArray jsonArray = (JSONArray)jsonObject.get("host_infos"); ClientCacheConfiguration nodeInfoCacheConfig = new ClientCacheConfiguration(); nodeInfoCacheConfig.setName(tblName); nodeInfoCacheConfig.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); CacheKeyConfiguration keyConf = new CacheKeyConfiguration(); QueryEntity qryEnt = new QueryEntity(); - System.out.println("NodeInfo type = " + NodeInfo.class.getName()); qryEnt.setValueType(NodeInfo.class.getName()); LinkedHashMap qryFields = new LinkedHashMap<>(); @@ -168,7 +164,7 @@ public static void main(String[] args) { System.out.println("DONE INSERTING"); - SqlFieldsQuery sql = new SqlFieldsQuery("select _key, _val, cast(_val as varchar(128)) from " + schName + + SqlFieldsQuery sql = new SqlFieldsQuery("select _key, _val from " + schName + "." + tblName + " where name = ?"); int i; @@ -193,7 +189,7 @@ public static void main(String[] args) { cbeign = System.nanoTime(); nodeId = row.get(0).toString(); NodeInfo node = (NodeInfo)row.get(1); - nodeName = node.getName(); // row.get(1).toString(); + nodeName = node.getName(); cend = System.nanoTime(); assert(nodeNameIn.equals(nodeName)); curTime[i] = (cend - cbeign) / 1000; @@ -295,4 +291,4 @@ static String getNextMAC() return newMac; } -} \ No newline at end of file +} diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java index a9d04d7b8..9b7c48c39 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java @@ -183,4 +183,4 @@ public void setHostDvrMac(String hostDvrMac) { public String getNcmUri() { return ncm_uri; } public void setNcmUri(String uri) { ncm_uri = uri; } -} \ No newline at end of file +} From 5a1ea86074cbadc8a87c329a4b10f1e0b25f88c5 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Mon, 29 Nov 2021 22:24:11 -0800 Subject: [PATCH 22/27] * Address code review comments. * Set index value inline size to length of GUUID to avoid multiple index page fetch penalty. * Remove unecessary logging. * Check only once to see if SQL index can be created. --- .../common/db/ignite/IgniteClientDbCache.java | 57 +++++++------------ .../alcor/common/entity/CustomerResource.java | 1 - services/network_config_manager/pom.xml | 8 ++- .../NetworkConfigManagerApplication.java | 5 -- .../alcor/web/entity/node/NodeInfo.java | 8 ++- .../alcor/web/entity/route/Router.java | 2 +- 6 files changed, 32 insertions(+), 49 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 8cdc07fdf..c5ac8a2fe 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -57,6 +57,7 @@ public class IgniteClientDbCache implements IgniteICache { private static final String NON_SCALAR_ROWSET = "too many rows found!"; private static final int RESULT_THRESHOLD_SIZE = 100000; private static final String SQL_SCHEMA_NAME = "alcor"; + private static final int SQL_INDEX_MAX_INLINE_SIZE = 36; // UUID length private ClientCache cache; private final IgniteClientTransaction transaction; private static class SqlField { @@ -102,11 +103,14 @@ public IgniteClientDbCache(IgniteClient igniteClient, Class v, CacheConfigura clientCacheConfig.setName(CommonUtil.getSqlNameFromCacheName(cacheConfig.getName())); clientCacheConfig.setAtomicityMode(cacheConfig.getAtomicityMode()); logger.log(Level.INFO, "Getting or creating cache " + clientCacheConfig.getName() + " AtomicityMode is " + clientCacheConfig.getAtomicityMode()); - extractSqlFields(className); - if (sqlFields != null && sqlFields.size() != 0) { - this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); - if (this.cache == null) { - logger.log(Level.WARNING, "Create cache for client " + className + " with index failed, falling back"); + if (!checkedForSqlFields) { + checkedForSqlFields = true; + extractSqlFields(className); + if (sqlFields != null && sqlFields.size() != 0) { + this.cache = getOrCreateIndexedCache(igniteClient, className, clientCacheConfig, null); + if (this.cache == null) { + logger.log(Level.WARNING, "Create cache for client " + className + " with index failed, falling back"); + } } } if (this.cache == null) @@ -304,35 +308,18 @@ private boolean checkForSqlFieldsQuery(Map filterParams) { @DurationStatistics private V getBySqlFields(Map filterParams) throws CacheException { - try { - Map result = runSQLFieldsQuery(filterParams); - if (result == null || result.isEmpty()) - return null; - if(result.size() > 1) { - throw new CacheException(NON_SCALAR_ROWSET); - } - return result.get(0); - } - catch (CacheException ce) { - if (ce.getMessage().equals(NON_SCALAR_ROWSET)) - throw ce; - } - catch (Exception e) { - logger.log(Level.WARNING, "runSQLFieldsQuery exception : " + e.getMessage()); + Map result = runSQLFieldsQuery(filterParams); + if (result == null || result.isEmpty()) + return null; + if(result.size() > 1) { + throw new CacheException(NON_SCALAR_ROWSET); } - - return null; + return result.get(0); } @DurationStatistics - public Map getBySqlFieldsAll(Map filterParams) { - try { - return runSQLFieldsQuery(filterParams); - } - catch (Exception e) { - logger.log(Level.INFO, "runSQLFieldsQuery exception: " + e.getMessage()); - return null; - } + public Map getBySqlFieldsAll(Map filterParams) throws CacheException { + return runSQLFieldsQuery(filterParams); } /* @@ -378,7 +365,6 @@ private String buildSqlFieldsQuery(Map filterParams) { if (needQuotes) sb.append("'"); } - logger.log(Level.INFO, "SQL = " + sb.toString()); return sb.toString(); } @@ -391,6 +377,8 @@ private String buildSqlFieldsQuery(Map filterParams) { */ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, String className, ClientCacheConfiguration cacheConfig, ExpiryPolicy ep) { String cacheName = CommonUtil.getSqlNameFromCacheName(cacheConfig.getName()); + cacheConfig.setSqlIndexMaxInlineSize(SQL_INDEX_MAX_INLINE_SIZE); + logger.log(Level.INFO, "Creating cache " + cacheName + " with index"); Class v; @@ -409,10 +397,8 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str try { for (String c : sqlColumns) { SqlField f = sqlFields.get(c); - logger.log(Level.INFO, "getOrCreateIndexedCache: Adding " + c + " to query fields " + " in " + cacheName); qryFields.put(c, f.type); if (f.isIndexed) { - logger.log(Level.INFO, "getOrCreateIndexedCache: Adding " + c + " to index fields " + " in " + cacheName); idxFields.add(new QueryIndex(c)); } } @@ -426,10 +412,8 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str } cacheConfig.setQueryEntities(qryEnt); - logger.log(Level.INFO, "Setting schema name " + SQL_SCHEMA_NAME + " for cache " + cacheName); cacheConfig.setSqlSchema(SQL_SCHEMA_NAME); - logger.log(Level.INFO, "cacheConfig = " + cacheConfig.toString()); // also make not of this cache, somewhere, somehow? // have a static cache? ClientCache cache; @@ -462,18 +446,15 @@ private void extractSqlFields(String className) { for (Field f : fields) { QuerySqlField annot = f.getAnnotation(QuerySqlField.class); if (annot != null) { - logger.log(Level.INFO, "QuerySqlField Found for " + f.getName() + " annotation: " + annot.toString()); if (annot.index()) { SqlField sqlField = new SqlField(); sqlField.isIndexed = true; sqlField.type = f.getType().getTypeName(); - logger.log(Level.INFO, "QuerySqlField Adding index for " + f.getName() + " with type " + sqlField.type); localFields.put(f.getName(), sqlField); } else { SqlField sqlField = new SqlField(); sqlField.isIndexed = false; sqlField.type = f.getType().getTypeName(); - logger.log(Level.INFO, "QuerySqlField Adding value for " + f.getName() + " with type " + sqlField.type); localFields.put(f.getName(), sqlField); } diff --git a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java index 980bcd9b9..b3898a921 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java +++ b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java @@ -24,7 +24,6 @@ free of charge, to any person obtaining a copy of this software and associated d public class CustomerResource extends Resource { @JsonProperty("project_id") - // @QuerySqlField(index = true) private String projectId; @JsonProperty("tenant_id") diff --git a/services/network_config_manager/pom.xml b/services/network_config_manager/pom.xml index c48b34329..57f523b8b 100644 --- a/services/network_config_manager/pom.xml +++ b/services/network_config_manager/pom.xml @@ -132,6 +132,12 @@ Copyright(c) 2020 Futurewei Cloud 0.1.0-SNAPSHOT compile + + io.github.swagger2markup + swagger2markup + 1.2.0 + test + @@ -173,4 +179,4 @@ Copyright(c) 2020 Futurewei Cloud - + \ No newline at end of file diff --git a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java index e366b89ca..484bb090c 100644 --- a/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java +++ b/services/network_config_manager/src/main/java/com/futurewei/alcor/netwconfigmanager/NetworkConfigManagerApplication.java @@ -32,15 +32,11 @@ free of charge, to any person obtaining a copy of this software and associated d @Import(TracerConfiguration.class) public class NetworkConfigManagerApplication { - /** @Autowired private NetworkConfigServer networkConfigServer; @PostConstruct public void instantiateGrpcServer(){ - return; - * Return so that rest API endpoint can work - * try { networkConfigServer.start(); networkConfigServer.blockUntilShutdown(); @@ -49,7 +45,6 @@ public void instantiateGrpcServer(){ } } - */ public static void main(String[] args) { SpringApplication.run(NetworkConfigManagerApplication.class, args); diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java index 9b7c48c39..adb746826 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/node/NodeInfo.java @@ -33,17 +33,18 @@ public class NodeInfo implements Serializable { private static final Logger logger = LoggerFactory.getLogger(NodeInfo.class); @JsonProperty("node_id") - // @QuerySqlField(index = true) private String id; @JsonProperty("node_name") - // @QuerySqlField(index = true) + @QuerySqlField(index = true) private String name; @JsonProperty("local_ip") + @QuerySqlField(index = true) private String localIp; @JsonProperty("mac_address") + @QuerySqlField(index = true) private String macAddress; @JsonProperty("veth") @@ -56,6 +57,7 @@ public class NodeInfo implements Serializable { private String hostDvrMac; @JsonProperty("ncm_id") + @QuerySqlField(index = true) private String ncm_id; // doesn't come in the Json version @@ -183,4 +185,4 @@ public void setHostDvrMac(String hostDvrMac) { public String getNcmUri() { return ncm_uri; } public void setNcmUri(String uri) { ncm_uri = uri; } -} +} \ No newline at end of file diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java index af6e46f91..27dcf4f72 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/route/Router.java @@ -44,8 +44,8 @@ public class Router extends CustomerResource { private String vpcDefaultRouteTableId; // store vpc_id - @QuerySqlField(index = true) @JsonProperty("owner") + @QuerySqlField(index = true) private String owner; // store vpc_id From af58b497e971604ead94d2e4d4064dd5da60036e Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Tue, 30 Nov 2021 12:06:39 -0800 Subject: [PATCH 23/27] Add SQL and Scan query performance comparison report and related artifacts --- .../images/sql_and_scan_query_perf_comp.png | Bin 0 -> 30559 bytes .../sql_and_scan_query_perf_comp.csv | 5 ++ .../sql_and_scan_query_pref_comp.adoc | 48 ++++++++++++++++++ scripts/sql_and_scan_query_perf_comp.gps | 21 ++++++++ 4 files changed, 74 insertions(+) create mode 100644 docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png create mode 100644 docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv create mode 100644 docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc create mode 100644 scripts/sql_and_scan_query_perf_comp.gps diff --git a/docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png b/docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png new file mode 100644 index 0000000000000000000000000000000000000000..c3fad88dbb419ecd11dab660ba1014bd44f69fe1 GIT binary patch literal 30559 zcmd43c|6tq+CRDwtqf&Kl9?hIN=RlZV}@jmgvgL0$vlTh3WWw^2vJB88IqJCQ)DQa zGDb3#A>zDO_r3SC_w(%M{LXoue~#DQTbpIA@9_Ct*L%9Y5qdgmbkuCr1OkChL;bKm zfk0kPAdtNxlH(`AtWnbVw#h}^^df<<`8DakWN~{o?;sGk2^xo$4LlR4KKq#;@A@r0 zLlJU?+nRX8`X%!r<_-zNw}P4KInQWwcx}U(S@o{2(eKW@weNKu{q93ogLa#!GH^M1 z?X%-JGAbxkWU!rFzafFTiM@vT7=w5!trxLMak^yjUZ~5(h~c2&`L)T>#Fd%SvEEua zJhzr2{-ap0;@a$%!`@Y+ch9Y_{hsW~7hr0cXwN#}+~;IvRcKQm`ytnO{a06&|I7f{ z9c_E!ki6m2UNZLCZDfSf)HZ@aRMVLQ>lt}@?_R%t{pr)nub-2{T`ohfBeJuzz0N(& z%bS}1{DO{-j)8$e-!wt&RNoU@nk1e4?CkOJap{1yaueZ&*;xk%hdW)nx}Ds=2-8wg z@EGXpGqSQu=RezR+4cD3zJ2>1KYl#coRpiBqoJ<;+-2}p7~PJ_-wS5GoF5L~r=_`c z+l{>%ha)cdbu(d{{rK_rs2vBa+jkr||KsOR+d`Aj(9rzyIqG`Jr^Ut19_bVm6xXj` zUvl3W7Z+F9GgV~QV$GG5lq4@Ne@I1T19{N9*i-Vo0DY6cFg!DNng1C zg!DC+P_?=<{)yN8Xltw9mMu353JN+qjqsK@68`@>k^xN$z5Ry{)m)nF{JF58nW^aa z>&N9S3NIfY$u}>ZJ5aUUG(60&G9G7av3t*+Jz`>A?d>b0_nhm5f6g_EaVh$j7Zn}X z)&1^KwUX4q`|H;)2{vt^48`9Qx-Z^$?S09;` z42e)(3k^MDVPWCpv-0e0R|YQ+vF82zxy7pW-`6>0*_c^Cp+?O!d{Rd$J9_Vx6qdTv3$MGSDXqy?mdkrTA-{&$rp5&uh$1apjKqRga!0 zwr{U<_$+?*5D^hc5H@{sajaQDH`&s3xMb(gBey5_Nckw|J=aN;o?Tn{Zf0U)a^eK1 zg0H8Ijm_fpXRQR$OD-GCAiJAv$H*u38fwQabJZG z{{HRTw^yxA;j^f(3wqqAB zUKH-=>+9?2FkoW3859&$S*Z}BXJs`wrLBAP=D{|`P7B4WzkeS%_YrR<%1k{!H%G5xXlQ7{a=CiIMH<`h z^XJb8&i58Pd2%%}vZm2!G`U&T_x8}x(A2kYxw*NNkrood)v7=Da6$PG5EQ{e+vEni}T~0ea%Y+}sH_Z#_Nw_#^&i`lmulE3J#Ry}sJ`;e({2 z;wf%=95Z)_ub*E_;z2%oA}=rR(W6JHHg4Q1I#arttNJn7r65%bz1=xC&jthpI64Zc zj1SdD?q|$;8WcoE9U6P{=H-?E;FMaVsO;xqo;$XEt?$5=fC>)ojre?vt zEhr{VPQnY9Lm8+;i9GcwLc4C}xTIUTxplmIC(Or3(Sj2_U1$X+IoA1mnVfo>uth9Lp9C^OVduhtkpf8AbK)Z zuD%I3YM+7v=c%+SS2iYZ-M)QdbX02ocuzy`@jIh3M|orF>guvHGD4*JV{z#o6U2AW zH3(W@+vgn&u7CF~Lr*Y0J$?RT=eu{bYe%@(t~OajS6ePEEm^jstPs<%FsDwNne9-? zE-bv@J`uB%ZT0=W;p_PEyK3#HSV-&V7aj~M=AzqDdYYQFr~R$s zMNbMz>UwxAj*n-TlpIJNnwZ#4Pvlhezm{cbHtZrD>l371(BmX^kAscv*WeYdzS4^u zd7ip|J8|gHA?nazddA3FaSEo$S{%qcbxnIaQ_G1mHwPu9m&@~Gg*R~7=6?QsR$Lrk zD#Cub`lXHtYrRT(`^4qTmtXq`wVPkbsi^R?wpN*Al}B}`GbSX)UNv^zEt()JDS7F8 zUxn8>t=L|%#wp5Km4c^FZ>_3lwKJ&j#@H_y)%|m?bM_K9ky_K~gu4tXZZe;%tF0E~ zYHDh*6cn7CyxrVxSvFl23H|i^!k0HuI~11&E`P6D?|b`JVBfyk`4+{uoet;Ei#}`a zF%dTM@+!5tT3P9qUS3tj#lycW8@`>D^<@iFHw$+18Rxd{?&oP~JSY+w8T=w5O;`*9 z0(K}t*!sjYpT+5>>yFsbRaO0MZ5J%8)W4m!u^H}sWVXKC5+EZZV`ON!b?erqzSeJ_ zN{XI5xqI*4#*G_MSMbGuK&VvKvygsjcTmrj-Me;C@z^*zmT_Ydog#C=63WResC)b7 zO~JhA@;Qxf+o}_67lQS7UJJM-EEAerJyoMN+DLA&g^f<>!%=PRlz=s_A+Lgd`}%d! z?i=aD&WkD{q4al?lizTPzLb`gU0GQ%HZ)92@gIHj_U){BeSl6Z>V}2)#Lgi)tC)_? z&hy;`%Hk(Smz5mF_I7r5s@>p}_x@pTY;26iVPevKSMscd=OE?gE5{8DfA;x&*Em=B z`0;sb>sN2z+TpB|lu!A5lcc(Q?AWnEY!6AvXMQX0Ige=R=$3!|^7iuj(A6a?Bg3|1 zN7d3$abZ}gdO%blP*0K zU&_LtY7%tXEYWYN&et^XtMcN8*2CtaU3GQiqfPfbHBX=Bi&8yZ)I;;4<|qPkCKsqn{wNl`*0efmpUT3U0nPHCocL6&~_7N#pT`^3djcisda-g!X1 zo?0orZKmk+j~{Nwk2C43%vu-3eUT+yOY1`BZWh(TeAa!n)V_~vvSuEWWd`q|Q(`hT zF*&TNIyy3PK{}Cchg3&bS3-RJE&+klRof)k9zA;W?Zx*>V1c570`AgVM{zJbJUpoS zzkmPs_xE>`smRaIfBN)kVIjlh8H-Za%qv&a&YXFGejOF1ZDEm&%KvKRncKws3ZF$3 zSaDWbl!mpnwWm)d&~jEIp7odw&o8U0s+t(t*=6PDi>0(IE-tDOtr*H)yxVqtU*F)`<+yOMy0+S=L|E?mG`nVz1eK752d!+_G| z;>FzTY$7eK#Qyz_EiG@?lEge`t-ZayuXDGReqJW0`zIW4;{=seH5{m|>zhcz{?R#eD`s-R6YG&jGo0~Rg(0|*~J zdK7n}#$2o3&-+z-dpjU@U0ry+4bDwULSmp4D8Q`P39STRY;e{yHdlWjrZFcc=h`*e zT{kt-^roh!UcGt+gqLyV=11NeZ?R0f@ zBfA5TUL%-|t4wYOMmhpucE>5vf+5CD+k|}J@)qF zOG}MH$KXlZGwsj2zfJwUy(vKHIa zxHOv9hwGWp^yK9$CMFy{9JW7q&TqJm!8Xt9v7Z`oD>E}H>+u|eZQBw+GZcK{g+x3f zIOIHCe*gMOs{br5Nz@xk1J>7mPIm1nm);lLP*A|n7qC#F=S0_)f2zOu+&whK!opd= z>3+{IQ7A7b9FyC#be;RkdtSY=u8-ZBYn;FKGe6+>KsAM&2S}1w`OGWdl09A8y?Pa}cz&!Uveuf5Dh$wvXHHRDHmfUAuN^p73&W zbF;C@ct=9+`|X;sQGkSJmVN8<0s{$(ii&=Izo)0q*Sm3Xah*DKDx|fe;}$iq0cWtE z-Cow+yLT_WEhyNF3T9V-q63Y18y8pAKyDoS^_%=dvxibBm2FMvVd28JU=sjpxQHm7sJtferKMi}{t60b&H)*{d`XD61*Y9HC$oP) z6BARBL;Hg}ced9b=q`TttZ8fDJJmyn4~N{lXQrx3mE7ImubZYfH#axxD1pW^>nXSE zrh|>mYwJS(u7h3v8b<($XlQ6kocqr^I?Bk&$<6m45^i}m2TJVe<<(}7s<JPlo9E`N=1{PWK+f(GLz<)jr9P>gwtm8ymZ_#Xm7VKI}%TT$eI`pGRgq z>w->OU!TR$-sTiV9L5Z_`~0Xj{)%RsC~gIc3cr8=3dz0OxAV+O9%p21xTiMPd$8-e zR=gp)?7Ns(cEl=im9*CWey$z5GPqyz@=2C*Q`;G%!T{ojhiy>8fRV<>Z{NKuvu~er z`8It?b8%O3HOGb7xWiA_`6($W84lQ)3U}h}*=3;)@P!m84{1oT5wBZ5hx?@ulAs(7k@+M*s5*>1AbQg@yZY(HrmU z3HmCesHRIwN*Wp&^*z6E8ff$N+1y8uYAcPueEH(q?<;adm<0e0&1qma7QMMBnpN{N zb@V+0&_-;R!s7cT6Z#f`AHu`w_*I%8*tqH!?CL$B%C99O_!xH?)N^(kojlnz=)%s<9&Y^HXYstEV%78K(jlz?55B&>dU~y(>k;&tM~}({ ztoeeFf~2XCzQ~s??uCbkH#aw<`Cv`r%H#gkB}NVGVPawF#K)o7Vz+~N9y_M|;OQnb zW1KLsDg=$WX+$vkCPi#=ucX4q6v;orafi}=n3eZGP+|ZrdmErWc z?8EEV8IK=d_YBy{z+V0QIgWJt!-o&(?)V6B9kI$K@+P6jrDtlER#r}xx`z77eLVX4 z5lgCmtT7uaZPhrTAt3#r=-XH5@iKXJ4D!5&S#$8km-gG<*FA>-9{c(e;XuUmydX zMrvvs*Vc7KbNm!NrrA}90pIA0r9xUEjW9RH_OdK&PP>1fR}Q3bXuV=oCft?A$ZKcu z`Ojsz?oM~5uYsboMHv|xxvwr=*s~`WKp8wC{4nqj-|E*_Q{0=}+4t;O{i%gLsH3AJ zo-gsv@L7SR5{3ORa#m)*zP9d=i)-YK0YxKp?k5k7x)DCXSZ1F);&g^-rm;RcK9z1 zUb%O`87#d;^5S2-aQMR!EcPI!nWfp&Q+(C_8nL}<;(P3hKnrChB!1LvQ7peWwpqpg z+s9{NVPW$<$9Sy^J=2~PSkQ_lL}sbS*P9Qs=(aIotLJ+83LIiySYxyYoi8f1sZO@U%3}|^aa@j$ zS8evr%LLO>owl@8nwH5t1F(FE_;YdbCap%Ka=LPs)pxx&@lT5er)!WBs@;ASP1T#-aK>el#t=M6OSkqDvcVGl{b#)DmdPtW> zkIk2R>{50Y7ZG`6lCIN3Pk3{@z(FSIu}S=JL*Zc@>QGV~Z;54`u{CcD@rKcP*6zt9 zdKEGWy5hIMZ)aQ_K;E0pp20c~OG%`E@9=0RdI%>3sE;;Wq|AJw2wn3$1IZh2k9T?LAS> z@9g18rZY=Z=jYuX%W_$jB3gTHa^K21p}bgd{_S{0H5gxwo_dF{ntJ(I%p5>lD z6;CLZjqO(!=C3#6zP{?ASNLT`ID^Xcg1h^|;-a&oqYBZ~%yqeXS%!BQo}dM3mYRx;a&UG&uy0?r zj+O))-MWg7uC7+|!ZpZ6CFehTPxqGMBmswm4<7t4#<+j)UeATSM5DEV>dVm5PZvG1 zsQ9%dV2L7BrMm3f)&j>5LM)Lxm(#r_-tQC>%ipOLFBreG)OAXtwX-wKFEKg!yuJO) zi(@k3SGP_D-?|MQ(EnS4^fllmg_MA~;d+$a@^ZPUw1CZ5pd+|_c|&)fG3E4IyMzzs z0jgucev$?EwCGiIQxxKMY}zN9aMs3#f4k}*0$ihFxFnn1A$I*1b{CrA@YtAEW=ToO zmjo+DCMH}6#os@U*KhY*||1#)0tNxIpylq^aSvCver52_#m}?exvLh!m2)S3o(sqmlo|;MxB1Z~ zVj7?tdJbBqZDahA!-t1QMjn|Kz4`FrSx(MwQBmNX;YTHXvAtK{aU=-sQAknj-^gY3 z=tS*^>wa-@6XAGJ>JumKqLscosNv5bEG&%105UAi-i*=!-m9gh1>#-!@yML4w9h1U zzy)SY+yF+#*!cL){{HV{V`^FL5RWyrw6O1IV|uY7{%_dztRYI(@891863k0)-ehtY zqX`=zir+a3xCTg$juoU7os`rx;_Bn=U0`1ReB4c@+KHGJ7Dfeu610U}*i>8jwoA2qxPo50TdM-)RDJv^4ef#x?I_~h{l>fsU2N$KfIyy{DNnci5 zOtfOXlF?f3EJJm4Dn`kIEVwEdf_7|SB_lJY8 z+75!~7d!O?-)QGv9+s(A=08=?Vq&ciT^d_HRE6gPE8fgahQr4iD)NK;{FdHcb{}?2 zUthW1yX%{qH~Tbm22?7|6|a}Nn?KC+l;c!r2<|veanbP63y<%3Z?^5*_uY91iiGP| zKX$|Ab)541|EUoasrzpnW6KI!{R3JvZr!?tUiz`S+j1upD{FUaE4QSssp%aco6g#o zHsVZ-j5E;?S7tsL@y8`3XgvxUAhNKqKx;wW!Xm|byL*?7Eh@}!8-HY^C0hA*NRB{o zrPOL!V^BGveU;D!OsIJ!SZb9Zsd~P0U-e<0iN#Q2xr4jfpV>M6qRUte3S`oPh)4lzj_1W4bNs3q9`zkVNHY6)1IU-;x~ zW@eUeQaA+kW8gt@w*x;ar7K?teTI3Frbj~4Cr)^eHFI22+koF@Z!Gr-t^=QkUjpb1 zT=Ic~20u?dR63-w45 z3dXNrHhcmm{BWOmc&u8I<$*5!zE>=QUx1qP+W{+KNjq2kWKpTBuI7=Jehy7o*`0Li zCT=(+9yA9zIbZi*b}4>m?CqDD4)j}cIe#v*wX_U8x%I|amFGXQ0JqP^V44wSnF2o9&#Orl7yRR>0cXzJ0sMz%=QMBkOq2O1!fL_5d$~I!H`|cmSf2 znwol`-}j~^Ly%G+y(>JQxw?zE)Jt87e*RP{G-fy$VC+1fy z+-0;Tgqw_ty{9LuZuNsgH@t?R-j=+Fj>yA#!_L`Rao@feprl24 z1}CW`raMU!@X8qJHMdM}_@C6sL@h4=5;1fB(+K%>4Ai z7uDcpV{|I(Li9)ruaT|7Bk*N`u(jX;M|)cOo>vM>=A<1FrC@G!(%U7jT=?!mN5G@V zytYnQ*D z0Fg zWjdg$p2~4O-Fc*$=|-p6lKYBIeac$}>WB~pSG)9mkJB=QmUA2Xb$CTnsnB&HYoQFI zgn*;Uaejy20&4?p9}O0}4vYmoEOtIc&g*@D|2oJuwiHA){8m6Gk-PLD-S+p!LG4$l zd9SEmexh`dEIO@douk@!!qX7Z1s_xrdKCzExT!GH1_uX;Y4DDr__=v_kakN#Vn4|T z+Wo$#r>DQa3UZpoJ-&7$BcWQhA))c)z${XIx0eCm)J8Z!)Bm-_K0(Haj8Tt^7i+R6 z+m++4`(zJsfi>K9AZVH))WEy&gf#x%>9v-2+OKKzr(0fAI zw{Pz$aZU*e0#*w$Fv9;77447HymaBhL7#=ZvL^%+!PaAR>cJ1mC4$~avYqd*{Py*$ z5Hs~|=80VH|JWvs{8;ejsrPFFsQr`c3r?}fa^tnkK-S5UqBe@}aK zznz(J2~u*I>=fB*7n7Eib%By5kQ{X3&k|#~(V6~aaU@^#SNg!$D>lVu6V9JJG}oLK zj;Ztih>_h9b%AcDVg0F-Ywk-Q$wg8-n4;gDbO^tCm3qsTl%&27A9gZRBgQeq3~3C8 zX-o`*;Te&cKBUW{YY~lk9l-=ww-WiE%*>I&nk$G3xMQEVP3YtJj%Tfv9P|1QDHg5u zsiGDXsH)}B9aCRGT_MDv0V?L$**y)%^%>=*C(^3`bb4L|P&W}~A#Qm1PQPz6`R5HB z9|HML<_i6_FKDUKOG^*#+7O-L(#h*QFE=~dt-L6*Z^bFN(8bw_*BfhvC99$04Avn<FdA0v)>L*91u#%Y3&W~R=!uALR`koYz6Tg z?hu`1xrwh|y}x$k00#8+^`Vx&JN?O~?9(lwUwo1q_g*>DZhu=_nB}Yr3!#OavvMQ< zgr(u2YiV}~G6%zIP%uW9w!O973(Kou0%ndVt;IPbJRZy9-ZaS0LBa+B$<$5*ry376z4;FNB<&{1#Wfj)_DR zmDIf`|FR;ob|f3p^uVWtit08B_eh3%s3t4;u|2TzZquG*(-xfXY428Rim+RF`W2TO zCNroN^z6vP_o1vpEUe$|DVGdOFG{tsu1@dxwT94ys)qAnt-S&ic8abOz&GE&2LK{Z zeL)6FRZT75qT=C@u=o#@Wu;~m3CZOrMVxZ3Ir}8a9|}ujJ;5UaUOvtB^T!Wxvwr`T zb0Mu;6jxtb?tKiHkfsMhf;u%ZG!#q^2hv*n4uDCD-@xBYGDeNfMe-G|fdzva&YvNE|%49YCLrt;}owOo*PP zB|x>?#fwEK<##U|?b<4LG3`QB7*6ob8&xNgV6Uji#naK=UgSE`Ff%iQWdoj%^n=vB zlcuH-AE-j2g^JXJN3jhBKRh(dUGgO<<1lzZaH(l%xa7T=@lpQ2QGYGd^upJBzRWMf zW&!h3a2t>IK5ZnlwcLmN@%=rN{_j#f$@bAzjnaN*x_hW5_o`G|^gc%B>?k4WpLM10 zXJEcTESvI1lFH-br%wnm8RuI(%CLVLv1|PoHpK15q2E7^=FLQL_ZlqsVh=E=r1<}u zc%Xd`TkarxbHXYAobte@VY{|WP)clE_1J)tOXiw0P=2VxUL9hgj*i6(cM%mq2noA( zjpDWgVWN3yMrp{x>JaW56+*o~qDDE!abFeArWztgr2Bb9kQwGXUx><~Loh<(*lxN9 z?x72$P2!*-d$$e!Ac4AB_VA*__~hhi3g?rMyX$Qr$)29r`#^lV%N-b#fkHXU$yA%K z+)PMVM8Fd}2_@uo1jOC$z=%Lra|eQrlA0v0(;DYYB8E4Ix(+G%H<1maO2K4d0(J_U&(k0$$5`yLA+V}z^r zI6t3@lhfVJ?WnqXH*6>^u?m!{HH0TAQ%H=CH)ebiW+tLT3)7!{zI~JgQb|mdla_{P zkVBIt^0VCM)u&H8?g1Nb6SI(zl>GeEfhkIL2Rplyz5U*O`y#@__1sNP5J*j~fE&f~ z*UHM|*RSJl-T)-NzUArZry;kxy3S$I0>d(>kT@n@0ESAhVr6B8b`M36mWprZ&P-s^ z*RMlXM}Q9Z&Sm&kc(Fi=?|Z~V zKYR4UOkTOEtALv_^W%qs<&ms3{Otx=I6tO6=RIRmcR}z%!7c2k^!pt zp`#V`<1#4)~JlA%Xtb8j0R0fI$vO%v{;zbnz z)M;YC_+fSIQCMWQ|YkxT$-`MZ9;4npw^&su|FXHT{daEu0KP(;P=X+)&q13 zD_DCh5Qmd0>HrCneWnjO{rcH;%1->w{Kxxm6lSyC4@ZST+=LmA>?JyT1>cV7jeIKP z;%>`>K6}OV4C}OXbShyE;m=uVRk4Sl-(0#Rxm>&L`}bmR)sWYc5)x25%747qYS^r@ zN2JdYIWs0-*^tZ8slHAmtwf zm@TBlgiQ#05)uV0Gr+LSCmrwYfm>tm`~_Nfrj*S7{23J<&WgC`j6_rSC-b_ahIV;O@ffbguf%!eevpT+;2^ zo&zt*aADJjw@jn2_k8{=FC*i4{(SS9B05@H1NzQ#OR-jHZ*So^#i&0WFm3pwhJ`@5 z>&!DvpB8l6hDwBs1oFihrD|4kKIz4a7svoZu|6TQYg9{^C0Cdx&@Qb=jF~V`X-HXg zaZCsD-X*oP7vXGla-8{4(Rp~%p#MPUhfxZpPe5P{%UUS%2JOGuU6p@-xXCICneP7~ z$IUGD38W$I&=u0E#ofcgsNXojCFfai?voU3tFvc)W(Jh&F4K%eGTyn zRlijj1p%rSZU`V8$|xGfDc@O2&X@H!1&~O0?tOmjgY5>Dvr$hD+rT7!{Pf%F zkOh%265@-bT`j179&LFHoJ96|Wt^;;+=&|veF;@%<2QnHHGU7~{wVn-yE8y)MUbEI zS6x!no9AJ%4~O$b17v-zwuM~2OmVO2jsIQ<0oJy*TRAv>*9)X@hg5=MA=`%p?fK6D zKrfInLNLrq3x;GqsyB$rIv@fHjf2C)C+Dgt+eas!H6YEiq)KX4AgaGfZJH%8J^f49 zsemY3G=Dh5Yp^>6wpS%5bKd!yWw3DUYNCRH%U7N#`%EXa5r;bm|Ss6be{ zG8}&eZ3R(NaLq{iAs1=SP5@4l=U!6xhbGP302*GBNYY99{qe}7j9{yb2sS!^ZD@E< zJCI_;xCcKYDk;1rB%}nzxXJM==wRJ$XatB1sbu3bMjGNEiC`;09syi%l6vvvi7Onc z?nIj8Be9(21=NJpT%?0+{a^R{Au)7=KGsRXlEl({>iE@|GurHn4qSksjFi=s{5w=e zd;ytwJ{}$_n}2o0c2S#qNcE7BQ73xqi4z|?I+)|FZETnUfaBsarwVM!E={s1PbbU! z6p*nSMjxS)*hbhMIbOx5Cnl0B)O~RZbQf8!=%ZO}_9|)gJQ7k;VFJySI(z?sg2|B) z!&9feBi!!fv^o?iCZy=7oi)0L4#H}R@0D|W?&e!kk**;x$bk`#FP(RA-XCa1;OfAK3-i6DPi5^%h>9?8!!K;ohK$UOjB zvlAyOe|^oBb)WJai8F=^rvUfU_H70SnrLb*VfcNI0P{;<2UA1SGPel;I%G3QijyKn zTM($}DL%Jj;c+0=ptq-|&6zV8vHBgbHWmOdOLAPFN&f>cpe+w#Wpc2vy!m6XbrcBP zJ^UA2z&_T~i#D)^d_C2rf4RqCW*lHv*a9dXzd}ExrK;)bO3=wd1vqUw){=r~1?k=^ zD4_jgnhLg1Y~$k-&#tzN0sF&`0_O)OLGy)4_4rhk1;>l~yE|6vMm|0}3obBo#bR%k zPEtct)9>jQ-*2&XT4!ac859tNS!U{XwkosD1?Ucb%KZ(X?@G#IcX>lGeGO%25f+Tm(x zkgUJuG1nZgM5++I3l~T;P}n*sm?WVd6++%~Mk!|>Dg8-I18>x<`~jc&pJQ~rQ0-x# zNlQz!?%1KYzB&)`^JbebX67>bBz~{2^{*dq*RkIUx(8hb(h4GIEh&mO1Qfh~fF&9( z2a%;Rr9u$QC^8KFACqy!G{mwLQd+*up6@H4n4adOrw^?7&B_Ci9vmEecV|XsX3izW zryFn&{)kqYf2*vo& zEn``knWSkCVj7Uqb4XBVzc&I_`Bc>T8NIx{hiJYMCHvdCnW;;hdU)gPk=W)F+ZCYd zqD%1)1#vH+KYY8%Z{M~OH$a$z1>o5U_Xq^4z+r0CPqH>!%CR!5y1DsrTAH%ZETkG* zszY~pNZ0h@*NF*SatwdyPaw*Ku?%~ABIyM0-o5L;P?REK`osnr4@TB*CnxVJ^_Ood zc$rzO6kaBPNca8pbO5cw>I`&opbm6SMn*;hW)8wf9Xr{F)xOGx%0zkJrRbJ11n17! zVzv(f(R3UwcK+#7++|`KqKimAQimenfA8+yufxLzu+d^-g6L&jhl%xcF8}aiW`OXb^OnvyyNMC>G+cy;V(TNH8t)9>^&YnFR(h5Z-|KtlP%(Jkvjt>rE z=B0XU9y3N1m)_>mg-<)QLWu0@QX~G0@>7&@-SEXlHhdg=Ti?)|?h=TJBRFA!U#e1gHRhvgLn8RB17f&tZeH9n;d z`Iw}nlSW1uFFBIlrmwG0uYz#8uBPTInJ5;@|3OsrgtacEk*YLXSa#bC=2;gp>;=KH zxcJ106TBf{85c(ywoYv|iX>l{bxxJSz2(7h1rf3XGj(XK^=?cg zw#SO?smb5-?_zPSG@Zc~v4FS=&f~g`=b19~bI_*{QW8xb`j1Mm3(Iz7IgARFG;~rF z9@VdNL-qYX7#bOUs_LiB4`~?A87EYoyjfMuW?^{C28n%)!@W{GVTyob;hgTI8+;V-2zaBdW0IHDDQb?(?stJRd7#J7`fQFd@-VoxGr?;v5Xf#>;;K6$9LMf+im$g+dnWW_Sc)#;x{Qt-Tgr&qH z+c-Wv{NpWV*GVx1b5qCejd906#ZhtIz&u(q_Kj5y6te&6}sKt?xU0xzJa3 zDU#a*Ifd@->ojpksGy?~0x9k6?ZH3F`uZ$_lu!bvM@B;J&h=NKlk$;H^B(nM3D9|H z%y{d{({DG+*j! zFo8gjll@ET*KVJ1LmFOg4<;Uwp283;4+gBGlacJYZEtIPBr&tL!&moz=MYZ`B+@`K zAK&*pBn{o8s5QR3ub+dA{47Ep*ZkmE6Ye!Zd)(Fh-r8|Gd9Rez+VD+X*NYcFef(H( zf$h1uXwPPzq>aXqxRr96ljR%F6p?%`zK{o5S$l+pf-helIpeTT@{|LV7Bri}r%y?t zpg4{z3qhmg+=mXV%aT(n8QysWa|L}NBjYusy8{O>H7&e*x5oK+mgCiNRQEk}v5X9i zjrTZzLR|}d1+k>OqN3*2D>-?2oIi^FnRDkv4$#Qa+}}foc`uAYHI%EWY#_XgY<4%e zy!^iD;oS0A3e&JCe)+h|mcb_Gek;?BvcT-N0z4uBTFF>SKwo?NP1W>?$@PBQ2#6GW z6pL%=kb=SkDC}p~&gU*Chhb<8;sAB1zTr;!ca#!r!zao=Y|f6|yymgt>K_n3-_8UW zaI7zkgv1J-(|;yL(3klxKQU;?O7VaA)RNG8u%+W0wc93xt*M*P3}Nn)P;3Ow95WRS zv>IB-={eMy0V@#3$+sqY;5Bfl75!J2re0%WLTEp~2qq;cN+j{@2Y}oShLm-g$T=KK z9|T9jiq?<7+6D?}p!xFVqq zQj5a`2*lZ-YGeKxbG}&Rhz0>B3DKNCcA0?$O|xU6Ps+Itt6N&8A~F@jrLeDCuh zZFXrFrJfiO_f~U)&G;w{WgvO#!@3Q%yc+U+Ds#)euxO!gA;{Tf9huvy?p=Vr z`qkA{kA6R6@!ZtZfu-3mh@I5iJjl!py>jJdJ?ZG(qOYme0Lo?VVtEF^9k381v_UE{ zeLYa25Cl_5`Hpf7n#;kVoz-4)?vrXFLF6{NRy7OafQ8+p3;_;{a%sZ#Hs5A_7Eg@-?Pn_#?I?+e?CBoJbJ@WYjC z{_Y(Ta5!d}o$l&nG1=tEXr_KkRb%#Jx*qPO)E5Xlsr@c&-sev7?vPFBW9ec)6?_p<$ za%@?&sfLCIJYNKbN-Z-ilh!~;Qb0h>)kOewJs@Bm(wP+(LiPJ(WW0f+;T+G@JfQTr zMt%}?$%EZ*k4LfRQ_eVqiY~QrQ5xSYJ7N02M;~4VAJL}F;#}UM3Br*DM2D}cssKspw^B_=w-M4p^HYioS$|ln0c$k5>9ERXa=ZdprG4#p(1*+|g@uq09Kf6HVYw3Fe%LcR~(x=I! zB;y-lRf8OU@Sh4Zpox6aA}P3NsE_$_a|b$yPMI}aqt;mh>!#AdgqQYwk z(JCGCSgkb>WUw)+tCe%|aD6TA9CM%;Plk7ULl`E;)uVdCu_Q_aw;jF&EUF}(uC6O* z8RsefamW|gT)-8<7{w}n#Lh`=3A}ze>m4cN^QU&+pWKLRhvp~*M-pFmuVuH z8y}-=OtZw_?W@{hrluP|Npv(C8WfgkR9u#7z+HH_sU z@(Hia$;rvh?FX_d9~k*tCA>3oWbtS!6U)Ph@>TplBVpI~2956aIkhI&#ua1A^rMbE z_Rj*pZ!PorUwxi`gp>ZFSeaT04~1k%SFR}fYqIC+(kvRt0eSgHQ}z-eO593Bw6DS4 zNETTavTAL#B&$y0|9fc}9Mwrrd;T2xo*ryX4DRwX@{s20IpsW4W4k-3J{`wd$x+^W z^PW^lKOgR}M@Cp(Q}ZK4T0C=O_BG;V2;~SV?rq{_iA2DyRR`{wkq6Q8Lf~ITnn98c z*$i0NsI!HbBb3_*J-)cuXgMf{DM?3-NVjFnpM>jXy#M&F`>L1q1bd&@MjG5tO8N}T zQT&j;;(srx%W>)o-d9ROLGbC?PwdrZymLvjId9>&_nY4z*zW3$7G)JS`!q$U9eVI5 z;wVq0ecPy~rq6o=#WqA{#1$3SVe7pn2Z_dGK2`w@;YyLR>4K8NThsp!3{BHD)Z6oSrPmaR0sE{pr-(URw)@sjGdbIjDpNACN7dFj@u~lWO-%hyH z5%j)p3LlSY#Jz(hipMp8QibS&;3GYRgqMm+aP0kKc*wb>~OYfjj}(U6#m;uC5q0 zP_ai@$F7D#x^WAqfxF2=21p{^-Qn68#l;t22m};NkfG`-NJ+)R%Di>!^_e0N4urMy zIxWcvmTi0iMwbK3N#F}n9+%;|7}Rp$vb8nR6x`-3>DuW#KL3*mg-2fvV%t&|NWvZ8 za)f9Oqzb)%U~Nhi^0bOuW#GYsm*!s4hzW|3pQHTa6dGtWeP$xUtw_UL_FQJGSo<}p zNu!}fN^g?MKlx_`o{tODRaoX%Q1C}m!JQ?MfwVGD@{Ot1LDNF_z_?p}e!Ezk5dkSm z@&DSou^qkgJd6=|P!h7pnN!?P8UHTI)MWzUkGU`(+F$N<-0M^4c_G$2H1=e^HU|jd ze^u#NQmn?94FwLc9}`WO8iB-Q(+TV75vj(J=pG?e)3K6)w78(sh!seIvKw;2=5e{0j@?H{cL}D4J19X3CS)|8~OFURO|lIXF%l9vS*T ziVYB2$q@1>Fv-J4#og~Ypn~U9nbiMLi2ntyI7+}lz(@&p7*>{+t#Uf)O0`NJ{>-$6 zOQBqGV_a>CXLVFGr0p2Wzxz?F_nSY3c3|E4``YP`(%%&W6$?+ky^uKYeDj5vBlMSc z+iob7NT)6AsNZ%>b-Q*t@ls4Xqrtwrysyf__uk#fnI7)O+(vus>O%=_2F>lRA`SGB z^cS}`*!ve;6N$V7BBe~A7u2k^Znj+YT=^Y95ezsu73HJ zG57EOqj&7^%r1@qE*}IIE^jxKekr6*APn5xdx;Hg24gMBUZZy~6d%BK=slr8*hP&+ z`jy`7rq8eTb7|DT=&P_GU8*IM_8C}xyeA0*8>j2Y)mViT(#$+Z`5Xj=gb)>zSx>`@ zlmuy>jxcgkRxndG^3-1Y;>zE9i&Av2j;~6I=a5w+XKa|Iav8xZY5m3c!E*=id4_Kw zyjCj6GKUV04NJAu4NA$L=&4c%8jPtTl0N+PD@Mfx-J}WJjzET+Dh-QBA1mts(L`U6 zS?&s>zNq>00t6nA18uN*&I@K!WZjpip$Ryg-pLZ~5<#y8IQ zt%RK)++L}=QjEdVv_m*@qDW9O>46hG*m0WNEl+Y5he=$JqrJ>R3ZTb5_!mc~E}hPxMAC+Z|eb%#56hBe(JMMAM_i z(KoA3td<8aN*jeF4(RO`aa6p}+gm#B;OHmjm3w-YOO2S*I3g~6(7kbPVH3kcv*$lI zd5~f*-6HJkBJm}<8z!EfyzN~u^1cMAo~GtzMis;}@R$*?lVyjA8k(Bk zQ-yD(*EVF};#7xStG;Pzo57=~7+`4`=zkLQ%Gg+qSorB128>6Sn2ZL43?WsgTlU}M zHfbMd>0etoa5dEDjS?|Uk+GaeMi@vl)(Y9lIm%(~Kx83p>Fb_L!9?qqHA*V)>4nv| z5d!(-j{Huu(!MN^=U`Ipl8NiS6e3BthqV(H;)Wfa5?CyK%I|S;35f#>hYJQ4HV~8w zNUs+TF|Upm#m`ebO^6YnF#1uTZ>3Gq;*!Z@;^G5Rw-oY;X%FjrNsiO3Mu1KqdZpY}gisrPT8BRXht22|5^@!h(WJ z?I~-o6~5si8=E$5Lc6$C@3KNg;C3UO;%mv>7$Hx%b!!!KRNNt}0~-}t+1W8I>5bG{ zh0YF6PIS6HMES6sLWsCB+{_Vz4$s}ceT4S{i|tN4pd}tK_!B|^I@rA2MZ}KE)ix6L zP;5Edv{e0D?A3_W@)HlO@WmqgggpE${58H1$SvX`kRRaLz!p?o9o!NcNkcHYP30Qe z=um5;Rlw)P?|jKc=8PCLlWk28{>V5Y_e`AiB z^rFTRL)%@dD>ceQ(<*=Yc9p6JpKyV>_cH9HbHZn+T*2ijunre=Xa(eo284eQ>aSgb zTk(=tur0Mq0uS*M7r|2oZ`Ir2q}CMhNC;FhuAvsjU#GSb2(KrtM%B+Beo8-w{KEg$ z)|ZDv+4k>`LW3|xQm95~lAVZb4N9^%2?50(yyr=hlf8XzM{Q9HgP-fS9ywyaPW14^wa@v7>n^DZi01K9n z1&K_e1t9B5nye*LQFa9`%yjT`8z5_gABwGRVIdee+)=rB=H$uoOUnewRJrMWy;N%nCZ^LkvyW>2mpmPm&dn%12^pl2DiNs(?DbjBg%m|r>t(BTVd;e z{rC$;agOVYf1jZ34tPxz^$D@DNb7Dyu(#B+4D37#9c2~qy57nlr(u0M-;+@Oz>h*t zq&uV3NFi5HT!V7B=K2d%J59PjO^3bcT>t)!P{FG1S3IQm$y#SjB%{HQij0iZ*t!+k zvnNlqSeI6=`_J#KH(pDk69h|p`?XWPkkaQee9xTW5w)i2mlm=th|sqbVT~}Ls0POk z^~^{qtSTKaB7tV4(27T(r#eiR$PR*1{G^LzUj0j^wtHaYhQ$DVrm+Rrh6&#w=ePg9 zpWyDL!Oke^p%NYGfWZk|TF`>L#AJ6^B-uPJAwf3QB$UrmbXY`v`m)8a2nRVZh*SUW zdyi5>$>Z+sN9-LG_B%K@z~AuLRSJs-AI-i1NisqUC5$0!wtM#)aq%Pe>&}fmfMgc! z9I`=+yg>LTI1Bh=+?EowIE#e@==e*RX1nM$B9h%Y3kNhpyM)cL4JJacj^O0Iy$MN; znL{=h&gasvz<&7bT{iUT9YhIaDnMajBRe^f>pM5zssm#R+CGRK2fY2!7kYi0l=3Qt z#`waz(-Z?7`~Bbzu5%w?V}Lh8M6C;BpF#L#KHHBZOp!x9}$%{$4-3y4J1oAhSuz-j`699>Gq>!j4w$ZOY# zh=XA}UTopmL%5D&AwA#^hAFq>u{S+f*=VQ5$+(Ll^XR6~#lXXa3#pI<5*cC3_uPeA*7`0%CJEzlLBc3;1er2`>lx+i|V} zGbIEoXF!Gv+k}5HM3o~xppm@s#P&u5N=&`KtxnZ{>$de1h*F(ga|UEK8d9AE|qV+j@HL+2kRtj)eAkly)CcSwY86;pDyL zDRR|F=9b;PcB7`TFJInatWEt48t(vAiGxYphn3pIF@BB7iJ@e4!vJtq%px&K05xUh zWhwB!JxcecCvm%C_N{UU=j|o9g#rRkdOVsk6yiofmjinR{{e3VUEM!H-^sC@y}1wCqRuV)Ho1~>f|FnDuxL5;@3q)~X zWVAW99+sx%J-vN>iqXa(k%nGZ?^I$ed1lGOjX*&if|KrU3yW;0?aAihHC5s(XVsp; z*u}TJXWX}Y{QH|X*0Cnix9S>?s_re`ge?q)OEq`h(&e#6%p5sIxe!jWCZ;)nM+u%d z4;5+zkeSoEM+V#>W(52aF1TJqZRZ8Z{XwX@V=8m0su|-V;4C(IP#Ufanm zi~o3rb&nstZxznJe3gO{!W zpEfX}q{cP$cxZfOa!Z8Or{lBuP@m*4hutos$IH69SqzgUedJhMR8+hHxuJ$I<4u?1QhonQRUSe9}>(i=El#<*q^!XX7fM0NY2M`H>`7~WtWhJjm%6?V7yFG=uNA!{I<+sfORjOW-=^F+CT$+5Fpt5jt&i&79=Gh2h6|SYr!MI z@Hz<27Ytfv61d6KpoLKGEBuR^t@ys8{#qY6Z3V2X zb;Q)fM9b^LGgR{OK9*IXtVu5%Z)fQ;s58Hw|7W|xvGX8bptC<(UVxXm8 zNe3AptT50Vhleynm2VjQ@0^hJprmAh1*ov!;w$y$2PY$?xKWGSY zF{?~#GYgAlnlGAFbadpSjWLFPuFfx~#X!4-j07v{v9bIP!c8L6x=uT7hm_%V#^aiC z9=zGimo}R`$e`V@NV*m@dHJ>xAA}f?O}}d~Y;9?A4X_6mnpZM_45tY)6l|Yb-(3F| zmNibDB@G;VkRqN6V25lLz5)gPGQCEMY9=h>MFWI8b_HVFzlJ zj!{%#{yeG-;`ZWDZ{L0(q*Xtd<&7acP(RGKk)j^z(IAb~0)tjmkr~FTEV$|hs-t}4 zMVU*G7t-PM71m_Y12+YCchkJK$rXoQZi6u)NIRgVZp-EfGH~(pi>Dk?Te}db1XmIG zYD3a%jpH zOLfDq*x!>`4CkeO#)yfK0WPdH;94{oHd;lRFh(tcG9tdzqjbBhEHv9!0tW&1#hSRl zyLPGyyw1ypAIscz9W=!SSO%1o4oenIR$cuOJaZeCDKmn@?s9B+UtChMS(vxr`Vnv> z+)A_nDLD!@nJz^MEKF;(1cMxefmm2?pgP12LMCNx48{?C*K>5Or2eD0N3cI0n8d&fZZF`2gBy+#CO+Qy>w2L!H6>M51J80N z_&30kPUQ~4|iV(JU^8pnjZqM!=7^ zNOG+s!`3v|X-XM{vcD69!N_6A=Cy%xCL3)`rUJ78>%pgs+sbUIa6R)BkU;ExdSBt+ z!^6(+lo z1A|LDGrr9>i~{E7$l@sPStlfP8t(TSmeGQr-P(cWfge?qUGuo_3AF@3w;Ps;dgGq0 zLowUtRsukC%wXCQdObex;3YM;_XK)*~+CIzx5=qE+x~D&D~bf`uS{p%M?K$=6SQIS!v<5E5hnC zVru5leW2T$sow(I1MkpCTYs79g%zX^b=deOnObMQ#2nigcp3_hn~uDH`<9!@2vMbA z_sy~dBb6-dPQewY-&SKd1}&V)wB8)XqMxFLuljz#9LFNU7-B=~T@7LClD z{jR2OLBfvgPtfq&p(*!Bnm-nUJg-+`6`Ts;(qTK$HtRdUEqK7Y%&|a2ba@DYC_ed{ zs><(oen@0@Z?LzmuDljQdl`4KMg%kQa;3MeqMeYa^~* zo0i?bA*_O9pm3f?iGjIZ1ws01+j z@l(;)l)DknLdPa7BxK+e#@v={e%bI1jKdjz+Ea4j(kTx`h|j$qnA>%%_u2iR)#jE0 zSNWP)AAQT=fAT3-1JW+YiBomuoue+Md3x?+Fn(x%DGBz;1YWkmZ7i`hCbi;1I<2&O zeAfM9VgLWDB^3Tu=WOL@5c*mSRXPw$+7-Hzs+#eBhV2n-Kt}6kqpxKYa6R z8OwZWaLyWbaqvDZ6N9(9qeWfgAQSs$DPPkV$PgD<&3%cLl{JXD)%1Z}#uH>M!|H3n37C9(2_f$y&7mZCU3oQPSJo?wBPqO&Koo*bc#c6&GaQnayH< z*n(}wWU}^e47`WD2PC-y&w$9Z4qZ@9qAACg5O9j{fSxC1HY*^eX7U)kOC+NU<8%;N zF^N|uX6`X2E|}j}w(#gnvETezP~jA}dwsU0K!f1frwwRApS}ZBd`2sINGBPE*cMo6 z^bLnl!`RFA;n$s4gC9R0n%k;Apo(=HM{$c@b$S%jOI5pS7>Ai;PRnC^y;!QHGZUP@|O zbqRZOO)z&SPKv=^Yw_yVOXoE6l{#aZLc7Or8%w3ciTvL#b+bc*J!)yvdQ?yQ*Ev71 zB?B@yyk{ds2i8p3a;`@;kaD80TI?@x+w^zga9NszxzD*rN$B8NmP%V6Ava-W09rdB z9Y}>Pj-trbH|7W5a)V-j@|{B^?1=yTCkLqo_+RuA!^jbhv^A zrUP$=*{nF2Z+<`P8ENz+T~fc6P^=M70l~Hfl=V{?~rji#zQ2)XRGuG1GQ%Fts=birTds zH@?qKdzE(IYjLgCjXo4&X2*xrQMj!M3y$fcW8h31;x|$=_#vRN&vB1m>E835qCFK> z_&(+r4Wk`0mE+Atw0!7FbPZNocE+&xagtKvq)yj3ar20rwf4G_ zt0BG-2xjz`D5e9~VZ#FQD~dnb$rt&1t5TICJ%}(|S++yUvi;&oSJwy%!0E9mSEyI0 zAzi0wHxoqO)+?U40BC4kSVzq|BLNrr060tmrE)1v)eCaWWek4)-0W$zul3?b%XY~a z>Ty!Aw84u)T1-hA4INwkz$8p%nqJ_q5_<_TF;v$^mkYAEN^I@;>|frlq2oO6v2k*W zyhc^ROW^QXH!ukTVP4z2ckiBHsz`qczC2_~n;*0nr_INeq(z2#vs3Sc((|IX->7N% zq1o(a#kk=1R?+3CE2gb2l{<>l0%A+jx(zDCMX)D>U2r(vJxUYr$JbS?@;BqVvakJO zdQP)qcqqQvkBv8tCTS;Dh`QPT5Skk(l;R`>ul{cr4K~wit=k_6JFF8Ywv;a*sRDb! zmI5LQrx%DS(B%5qT}8R`ZGHBJVP-Q{Y42WDSQkNOmeJw|$GyQ248KA{x!@#*ko^O$ zPiQv)KXu@%=>lzM#B8+gUG4%uNBL9G<}=mSCYv8aHizux`*f=X zpG34XDEDS&uV8C2Qfi&+1B?tO1!n}B8)I|xl*Gi*u`v*6DC*{mMmu+|qyzA3${+~M z*|?rt=eJ`528QOuYH@?kBrcN8eLxJT7;n`H|4Xuhgd2_<-vEL+jObNNs$&7CW?*h9d`Vh_eQcSy25Cg_7y1FUJ8m=Yw z%I^1t_A$exc`RG$N%Y@~=*Q5V;{Lvhk?LOqhAqNyt=ChI0ABn?F};kv%(mPE)#agDO~&&WDz-jJTU0TA2VK@X@JYC+pmpS&OPMJtQV*2&d#u>>w-O zF$Ij4mv=R?(9hVEq@VP=MGZ(;|7tXye)OnJdzC)|L(uc@;wQiS403srAG&|dD;RuL z@3*o-ET9-JPI6ly2%w1mEHB0KQVbv1su=c7T_}e9N%C$zlQ>o%c#|{$(h1SgD#e#V z7QigKjbPTf?K_W|c3I5oqOP%-j$&K65Xxo4belXLD$))_Ktr2EiBIt1m<)^5oij$E ztgfD5ngw<*J;GFm5a$NWs{A^|Emm(|D8Dssa_oZF_$?zITLC^~rHLwcbx_&L9qvN` z1u5P`#3y-qvrOw2r}&SbKAFk1!X6wv*C8LRPyzJ_1d;3K6l#07ZFpU=;~9H(gEyf^ zo6g_^|2y^0o#X0>;L@VWe1g6>btrr^tY(+oiK-L%0sS}k05y7$nqg)IXWoDCwthUS zQse`_FC!a%jd=+VTU@n$hABTACmf}kQZ|+$8qj3r8kq74ds;UAXcWttFjMXdk!K%j z&QIQD$mEc6>@OD;sXG^2qmuZ_yDaJH*rnHED0{WVYD!RFN>|Sh@b@PlI&0^qnLl8b zO%xN(_(HqLJlY%~-^@m_$q_;a<^OspB5Z_83nua0I=a-tpU+2Au~#FgwbO@+y0)d1 zjg1H+(olf*TsP02a$nIFW}vR%B2R0Ht4Tn&-O!MsvF6IA+5mEu@$XN0SX6}J`khkA z_g*iFJ|yA_8itVh7u+hp1Mu zHIw0oulPenz;S8$O6Knj-!Ff>hLwXYB%PuYF21)B?h`0J9~|@~`?3?y25ki~0SSR! zSQiN3TU=*cVRH07+uH9w~ux-zcICCE;c`o`pn{=)L|1KF*+rEcQ+4Y%sx zFpQjzS!hT^%zlS@=H-)3{Wo#CWIaEy*kuWM6}EF;n@QiJf0A?|LqN^lv3$k%&Vj<` zWkXTt%=k*XlhGRA?{MyvuD*D%otTD&P`)1lQxZ&G%<;RXcA!nN&P_^4_^7dy>K|LP zvOmRuJML-jp~u{x_6al$@if@}ewj!?-3D;P^V&dydD!Gr-m~_47Hp&YNTHwY1okAI z6VTy$E%w}>UnbaPw{o-^N&h8NT^uN(5Pb=w2oDF_2D)%0i|5+r@X&`MYC-;&D(PI1 zlt8H2Cu&gbof(f(e1i|{7*^1snx0RF#Xx&=^Rr{d)#(jPR(|LvUo$?daq0d;YxbHF z=j;h+rDW>+^-UXL#~i-l2c7<4PA8{^o?dEdY9BkFoQ7P#AOi(9;g;tZ@U0eOBynxO31Au*IqOt?!<7sO zl6ha?KN3B}j17qczzvY&Wpk&h8ay!ANv=HbtOL*myuM(6Y@j3;DzL0;e;NLFtIkl4ngjz^=GAZr< zWHcGVr9f=cn7OpaNV;4K>;q`CJiLZk+I=dsN~oSG&zq_1+F;E-%+KlvONJfUNAK`T_8A+}-nWpwDt2 zoT)gO8Q-NnSzu7VA;@PW|7xq^dxzQc}R??R~X-?cG9COlgQw?Dv-h~ayMGuzxohW>7R^$OjEaY0AJz$7MQlatqwLj6_Q3Nd zaQEQH$$L~UG{k7g_yLc@m!)eAf-7~)-!4OtvV~_HC-wQz){}7!nU*y4z_crGT-6}| zMsbKr55>e=S2ZwmSr*&9KS!8kVspXZ-}U%$^mkaPX)0!I08wpiT_4z@SAm=yS|yww z2pz$3-m_R<^#;#fyL&g@R3>Xe{+1d@e6DH#ZP!PI@xB=ASTL`IHW$?wYGdF|EJmw$aBuuYKx&Kwry+fi<2Bz*> zEg~Xy^Vhr#CJ@Ko0n_R6_y^wyU?aG|I$ssKx|4%{AYj>6b%|EPn_Y-_})bE%AMnatn^#Su8 zA&+SI!7oU%S@+b@f?Ni6&Cg~GFbU>usy^8&udR`a+d)31l{~Hes&@)dsgWn&^!!`c z4*+B&9?_$tGmg=U#~&cB26tlARdIgF!dYM6cGcx;Z@rJ;(AfnNKu5oq(l31ql*%hs zRP{Q#7Ij_Q`3V~yqjErmz>sc6N3*akG3g8j30W{V)xT$>x`v08Smo@jw>_~=3@C?TqH%UIT2@)34Rz*!qa6ADQyE|zCWS1{0R~hj!%0>$5*(& z0!~K}tN<7|KmYCM43-;*9VZFk7ST*RU5r7Ac-hc<6hP0ZeDp|t^XB6~yVMiMGa_}hP^|rVz<`8(HPxl7s`|-C3$s)s zT=%cu4^yRKJms56!FfP6M6d5e^TLotKY~I@XT2AIGRR%)-yZKQIu=>epC(w=J`4(HXZ`0Wh@B)#33?^`TEOi&(BS_X=fOXfLxm5lj4WY9o0OZ0 z#%!BS$XLQ;m;az>=z}n_3tc^%hjd~7dub(}#)CQIPWC?}IkU;;zxhF=ZL@l*0&C6& UUy1Dlse`oJ$lUNg-TC7G06m{b(EtDd literal 0 HcmV?d00001 diff --git a/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv b/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv new file mode 100644 index 000000000..6e4a5a479 --- /dev/null +++ b/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv @@ -0,0 +1,5 @@ +OPERATION TYPE MIN MAX AVG MEDIAN MODE P99 +SQL_QUERY_EXEC 1.0000 16.0000 1.8739 1.0000 1.0000 1.0000 +SCAN_QUERY_EXEC 530182.0000 785735.0000 561550.7678 552705.0000 543704.0000 531308.3565 +SQL_CURSOR_FETCH 0.0000 1.0000 0.6717 1.0000 1.0000 0.0000 +SCAN_CURSOR_FETCH 0.0000 8.0000 0.1411 0.0000 0.0000 0.0000 diff --git a/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc new file mode 100644 index 000000000..8e9633b5a --- /dev/null +++ b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc @@ -0,0 +1,48 @@ += SQL Fields Query and Scan Query Performance Comparision +Prasad Kommoju +2021-11-30 +:toc: right +:imagesdir: ../../images + + +== Introduction +In Alcor there are some cases where queries are made on the cases using a field other than the key. Since Ignite is the KV database, these types of queries will require full scan (linear search) of the cache in question and this introduces big latencies as the number of entries in the cache grows. + +Ignite supports secondary keys, called QuerySqlFields which will use a tradional B+Tree indexes which can answer point lookup and also range, minumum, maximum, greater than, and less than queries mush faster. + +This report is about comparing the performance of the queries on non-key fields using Ignite's SCAN query and through SQL queries. + +== Test setup +Operations are on NodeInfo object. Benchmark tool, DPM, NCM and NMM run on 10.213.43.161. +Ignite is ru on 10.213.43.163 and 10.213.43.164. +Number of entries in both SQL and Scan are 1M, 1000 entries are queried on. + +The first query always takes way too much time compared to the subsequent +ones and hence ignored when computing the statistics. + +All times in micro seconds. + +SQL_QUERY_EXEC time represents time required to execute the SQLFieldsQuery +(cache.query() API). + +SQL_CURSOR_FETCH time represents time required to extract the result from +result set (cursor, cursor.get() API). + +SCAN_QUERY_EXEC time represents time required to execute (QueryCursor +instantiation, cache.withKeepBinary().query(...) + +SCAN_CURSOR_FETCH time represents time required to extract the result from +result set (cursor, cursor.getAll() API). + +== Results +|=== +|OPERATION TYPE| MIN| MAX| AVG| MEDIAN| MODE| P99 +|SQL_QUERY_EXEC| 1.0000| 16.0000| 1.8739| 1.0000| 1.0000| 1.0000 +|SQL_CURSOR_FETCH| 0.0000| 1.0000| 0.6717| 1.0000| 1.0000| 0.0000 +|SCAN_QUERY_EXEC| 530182.0000| 785735.0000| 561550.7678| 552705.0000| 543704.0000| 531308.3565 +|SCAN_CURSOR_FETCH| 0.0000 | 8.0000 | 0.1411| 0.0000| 0.0000| 0.0000 +|=== + +== Plot of the results +image::sql_and_scan_query_perf_comp.png[] + diff --git a/scripts/sql_and_scan_query_perf_comp.gps b/scripts/sql_and_scan_query_perf_comp.gps new file mode 100644 index 000000000..e63b95324 --- /dev/null +++ b/scripts/sql_and_scan_query_perf_comp.gps @@ -0,0 +1,21 @@ +set terminal png truecolor enhanced size 700, 450 +set output 'sql_and_scan_query_perf_comp.png' +set style fill solid 1.00 border lt -1 +set key fixed right top vertical Right noreverse noenhanced autotitle nobox +set style histogram clustered gap 1 title textcolor lt -1 +set datafile missing '-' +# set datafile separator "|" +set grid +show grid +set style data histograms +set xtics norangelimit rotate by -45 noenhanced +# set xtics () +show xtics +set title "Plot of SqlFieldsQiery and Scan query performance\n\ +Input 1M entries, 1000 queries (excluding 1 outlier in SQL data)" +set xlabel "Operation" +set ylabel "Time in Micro seconds" +set logscale y +NO_ANIMATION=1 + +plot newhistogram "", 'sql_and_scan_query_perf_comp.csv' using 2:xtic(1) ti "min", '' u 3:xtic(1) ti "max", '' u 4:xtic(1) ti "avg", '' u 5:xtic(1) ti "median", '' u 6:xtic(1) ti "P95", '' u 7:xtic(1) ti "P99" From 02f04a50df336f764bf1821f559b540185d75151 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Tue, 30 Nov 2021 15:15:11 -0800 Subject: [PATCH 24/27] Address more code review comments --- .../ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc | 3 +-- .../com/futurewei/alcor/common/entity/CustomerResource.java | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc index 8e9633b5a..b387046a4 100644 --- a/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc +++ b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc @@ -13,8 +13,7 @@ Ignite supports secondary keys, called QuerySqlFields which will use a tradional This report is about comparing the performance of the queries on non-key fields using Ignite's SCAN query and through SQL queries. == Test setup -Operations are on NodeInfo object. Benchmark tool, DPM, NCM and NMM run on 10.213.43.161. -Ignite is ru on 10.213.43.163 and 10.213.43.164. +Operations are on NodeInfo object. Benchmark tool, DPM, NCM and NMM run on the same physical machine in the lab. Ignite is run on two different physical machines in the lab (partitioned). Number of entries in both SQL and Scan are 1M, 1000 entries are queried on. The first query always takes way too much time compared to the subsequent diff --git a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java index b3898a921..980bcd9b9 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java +++ b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java @@ -24,6 +24,7 @@ free of charge, to any person obtaining a copy of this software and associated d public class CustomerResource extends Resource { @JsonProperty("project_id") + // @QuerySqlField(index = true) private String projectId; @JsonProperty("tenant_id") From 913cf6c541f52f858cdc2e68152b6ae97b9fb2bb Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 1 Dec 2021 14:19:01 -0800 Subject: [PATCH 25/27] Add insert times, remove sqlColumns and rename sqlquery and scan query as suggested in the code review --- .../images/sql_and_scan_query_perf_comp.png | Bin 30559 -> 30122 bytes .../sql_and_scan_query_perf_comp.csv | 8 ++++--- .../sql_and_scan_query_pref_comp.adoc | 8 +++++-- .../common/db/ignite/IgniteClientDbCache.java | 21 +++++++----------- .../alcor/common/entity/CustomerResource.java | 3 +-- .../pom.xml | 0 .../futurewei/alcor/scanquery/scanquery.java | 0 .../pom.xml | 0 .../futurewei/alcor/sqlquery/sqlquery.java | 0 .../alcor/web/entity/port/PortEntity.java | 2 -- 10 files changed, 20 insertions(+), 22 deletions(-) rename services/{scanquery => scanquery_test_nodemanager}/pom.xml (100%) rename services/{scanquery => scanquery_test_nodemanager}/src/main/java/com/futurewei/alcor/scanquery/scanquery.java (100%) rename services/{sqlquery => sqlquery_test_nodemanager}/pom.xml (100%) rename services/{sqlquery => sqlquery_test_nodemanager}/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java (100%) diff --git a/docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png b/docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png index c3fad88dbb419ecd11dab660ba1014bd44f69fe1..cbec8d34dc2b72519e76874a71ce2f90a325ca98 100644 GIT binary patch literal 30122 zcmc$`cUX`A|2KSAMkv~gQqfXcT4-rcX~;?wO;SmFNNH-)RvMy=P!ugK6_WOr79pvW zc3qD*pYQkg{awH7I_~?Q`?%l7@%iXzyx-^f8qeorz0S}hht;V!vu!2_f?88U#eg8l zO9_Ikijo}P31GP{gMT(%)HvZn5L;_V|B>Ak+`640I0;RaLq_hY>t&e@FqM@M{QjCyP?`9`G<>!ard8EJn?)uS1=I8#OF3ohC zgZycfD7hp!&L7?Q=c`8l>l*^`EzR}YsVTVvX<|1^v5|gbH=~0DCw`!@r^`kWKZrJZ z-2CSca?P{I@Ow%Im1-3Dy};~%{}0|E+VbJ)5@YSaprG2fA{E}FZx}P%s;Vr1ek=<+ z{`1yBH{ZFE0ai9P4w;MQm%iT7PB4CW@~pkRy}7xl)$54%`^VP4o2|v3dvfNLrALhK z;#a=t10tGnWxw7Z+`=Rj&6~Eoyo^6ePD;{EkvjiAb^p(bwblL-w>u|-l5f8^`fR7P z`x`r1Mrf59g@KH4!l?zF3m1g1R!&Y&zr6G{c1By~(%6R_-4x!!mcG8eQ$-hUCL{z0 z28J-+lJmN}Fg+^7bpQ74+m9Yij5NgU+O_L#{NDR{dC4F46q{VA<=?S&BjN1qyu7l) zS17?+y6`z=aBxsqSeWl{D!+h0Xjs?**KciAL0kOh2RAyijjCi)PUqzaL|UpVYHWC! zx0Ps4kBMo{R1FUqR1IhH@bWr%;J~?a=T--|`9(W#YHx3+KW(e79(?N-`wLV37cX9z z2+z$=4cDmi@$qfj7WL}YtDKx1vAYR1_4Q-5%w{_ZY<~1Sqo;Py+{%8y=s4*$|9ZP4 zja9txX*LFik%C^XsR|tFyDS zCr_UATlsEQd#<^orB5`|BnE>2c&bDQYfZ~Z=X z(ZOenb@{des@^5SwdGNr!Dh#xm{0Q;A0DYmM-txw+~m0B|0}V z^T4d)V@*v>N5}86=B&8xPqBG=$+y<$Cem=JPXGEf{AA-<|gi3kY`v$M5j-S>I)Z9;E%_faDw32EuNnwqYzu6OAQ>HORj9Q#CK zZA>?9+VuMM>!6^Z^XJcFJ)4WNV8ATn_L`OVef(ISnVEU#4v&;nM^8`kaVtm1{M=jt zT03oRZ4;9nS1ZTH#%%2EDg58Pd)IuQgM|fueDLx(&8hXOEFB#k-tvCGJ~v{>@RRz6 zhUWJ6GQTx>{hrQFYX=8e&(r1Q<>AaCj~+h6C+q6zZS>E~yA#oUrg#;&8hB zUWWbIu9q&2efx&p8EZJv^`yG1D{+2p^z&yXHnszvgA6XJYHC^s*fs@KrKau}8ln(R zDDn7pZfGY{#tW(`TRFd-Ujnx%us_0?TT z`>a*AImdQ&cjMz@w_I{_;|o_(Rz7|DbWqdtABjnkng)`tuC8CdfB!NvawK^7StHAP z6@--(8lnd_JLx8A&YGdVf=`SUpig^KlCn3m@%*4F!@M3WKe zO?-TpU0vCwMFK)YL%SrdU%y^fR%U8yI>Lg=h=Jd@jU(DBL~`0)wpl2IOgQ1~sPol; z0QtqSte@peTZzw&DR=R@e}4;sqHeW8N?aTTN+JXGNVcAvo}M0`7{{kax3ejA|L~6= z=PfM6KEKr1u{BtxkVZwCExh*I_;~jFOFFT6xwr*$bH&0)UizCXUn_{+>(84uznJ%4 z{qW(#kKQ7_LSg;k385_O-O>T5;A84c;p+*Ec~&QP+^X>-^mbA6C&3^Cf?dl?4;d0g0C1qr~K7PDlZ7rQ4TH{PbO?~=>^QhnIubuc^ zZ2AWe9yANSbQ*r0Cg-(BQ1IZvgTdEGFO_3^lDED%!7qMC^ks5Ybv4VmfOqCvL%B^yOl9_lf8?7T8Rw=!n0X z6!he%rxZ@k6m6Mm(x34wD*a6EKR9SH^qD*$E-o%V-%($mR(46cb6`L(K5W&Q?yQ4D z>6r$)Z;Z4tspw*ZRY94BNcdFX?{5FJGGZF7OBoN7fCOO}`POr&iXvcyTh<@Gc5h z*@gZC%F4=5TE2bz_Wu3*8#fq$Gq7OPqd3|-JEKY@`NbKPPwUEZ=$VoXP>v@9$v zJ9iFz{5UZ@Ty6OTKZ@M3-#}mg(4~D%$&zpT`{nlU@2;$@Y-tfkrLQ!vPfZ97ZRmJ- z3QJ}Gj>wdhQ(j(P-rl8XSRQpfW0ITKk^A`gWZo+;|Jd7mij!u3agm&$%}7bvPB`=w z<`x#B5)@`;Ry8#>b#)b&m&?n^mEmq4JkYr&S`&EfTF#w2{a8;kUt6=ZfBg7?I=p4` zW|ZjU;<)yF zhFSOTfB*C;IyP46u(5$b)wzOh zhu;gozSPEZ+CFmnnTEppxUjITv$LqE2!KIEM1)T}B_$<6#O%GfsL zup~4!HL>ttzI=)1jd4a>wu}pXC?+Nr^LCW3b!2pOYQftI{nyaY&226=madr*%?^!+ z-ntFrla-lS9~AyDfSYP-a9nh>=z%Rjkz0dz3DjYRw+5qG_x1M+3JPj#X(j6LKFO$I zV%ff3+mu2t=Bo`ud)vUkPI`K)^XD7QMT<>@J=Cu{=!a&^lS@cSdU$wXxVpQ#vN5x~ zymV$17&obuTZxG*jEuFGPa+~Bun0Qa+f^BKZBA!a8jf3LSu<>>;4UhokxooK=yv!s zsXy@>DBCG%^E2cXcLzj!dVRU&hQ6bc-uU1GkgK(|HS^A$Yrv>)?2}MCFoh@x2i+!a zbc##f;4UYIFPLuG5)Qo6H?PtENs^kO+SP zt*or*>+4T%rlY<1i~IUDu&psgPMUQaHmIqq(_4q~72NCVGvSEpY-w@Ef@#)EQ;Z8t zVYZbRPy5T;CUyR=Jk91p=cCiHY0`IX@M(?}E6*b%-(R8Fs8W|tVqeE1Eo4L;$3vi6k$>!Wrm#1D&~ zrJTOy_VL7H^DFNP4729N>l187;+mA6^O(H6WZRy1^!fAW0P_3Jy*myxU9mbV^28Q6 zHbMBb-MMp8=X(nGyL>KPTU~Pd`H@E(mrJU{I+w~aC1o^sdQVn@3Z=<Qwyim0;TQUf2Zx4QdU~`3+)r?_auVYHAi18F@X=tYRG_+YVNZXf9sU zli$J`fARNtH!yL*@dwy+SnTNkkr5F>r=CB^%L_~W+$cRjwQZZ5lhb~uAr<4)n zHlsRX()VIueoak*Fah{5uva*Ai#s|x5-ef|O z2M!)Y-+Mbe6~dmrc;bGn#)+#2tIGU>f~_4LkM7@3v0*VJlVY* zjc#Z6?fduMj~@>+Bqb-GJ98$w>A~!;U*E^ZIKv{YUvCI5-5gYfQc@evyd_AfS-K15 zTVu2NJ2hnujgb5I?F|g*_CGBvJALe!w4B^&gDeXh8-6jd=8qpu&CCXxoAoax#ZJ?O zN7q`OJ$J7BcaJ?L2f(LlK;H61OO=RO`S{nb4QIsB;P71$(^CHaYdd$%kr1Hbi4&~* zKi7oPqZ{C7PZ>iWf-p4J)dhRcJ9&G{E(D&_ys2@kM5%~n>Tq*>JkvMP!)da)_wQ@_ zRz5N<%X|3H=UbaV#t|7&QJU@B?X9d(hCVl?r^m%9>+JFzzj32@?75bP2FowaL8UjR zbDljz3ruiNyMzl=QX;SY^*G2tf^BhW31ts028`v_E%3CySFZ?HCd+|~&GmP3bDL)c zwch;TwCjKA+SYhNUA^w6;IT*!GZPakqDFb>j)TOJHghm0MMZzquc?S{S2u2>8?Kc~ z$ZnuSOF*F=z9zQbVy=cE6X0Neau6K?%!iqo`IwOrusFRc7Ag4^|AsTB6~1MUA5Wqu zV`%P}_Mps%uH=FA^+}tBqeO0^q~z!4=d5F3Vwzs?Ml;kb*v-Rp_RJX+PDyELJ|3Rh z+FEd7!!JBc8Rg}^7cM+{_G~tkp0DQPAs*Yoisk9>{Ctu0ru>HwALi$Swa!dUdEkNq zV|gScCCU7+UZvRoy0X#&Z6_CyYf)OABbn_s)D~1_q6SX96lJDkC5dXU{$>ETmi) z0O-no@bZx+?nZh3Q!8N^U0XBB$;tQZ+4KB-?`7uQZ3*z!uF)tXY3U-TrrGNrIb1R_#m}A{J$0(& z<;!H2uY-dzX=#VY7qXfpW`6%h_YY4?3JD3}H1k!CNw5h738@_Evn%aS8gw-7x~|^dxQepfM~8~H8#|qH zKrx}ErL7TlWx+nz;oT=KJqInvL27ArWl1UHF}8}CS-NbA=mCwen$YvA<-VHy3qwHymSPTu^JlVS7H;$oMB)a+NMZ~Lgh05J<-)gy@*Xf$#9l*4a)F3u!f3+>3~ zw|!o1gXttmX2m8GAYiD{4ctSA?x?D&TD`Y%bktjZn($eN!j(GgNkPFbuS)*av!RDk zEwS$@2}n$BF*=9bb1dwfTwQsBPY4-XTk|b6S65%%@%h&RKng&Pqu!Y+kGG1jg6Z*oYpQ(&5oCpk-E%NJ+pYSR4tV_8x1ixVY33GLB_VelSOC3sF-> z11u>y=#k?oVc9u5J>BBT%g;|q)Za?w7SYe#uT^mIFHeNR2%Z?Uh53E5dBtt8g>L+OX7!KH*MkFs`tQb!Ofrbh2=)Z0_UI7lpMG z=nw7fjO^^xgvxfOcj?@~Uci!ZTY(W=-+#;iPXLK@ZniemKuvtT+_i3;QX;O zac3@ED0d!xL(g~E=-9D*2ws+298q=Vq9`woe%k8l*3%Y8gq8t#0oAy9Q?QL7cuGU4 z$m})UP+;A3`O+nZmAf9F-?8fHH35?~81H_0aY*!tp`o((4h{}s5`Ew~n(46JSoqXC zXo9kH+U#makKR<|l<}UwXl!ik>3P7ZRUxXbwX?IrdqKRNDIt1(>^#rcx_i589vw0< zAt&_GWLc%@?7Zd8@REoK3){>ary02Cm;AjC?CS5(9HAycrm1<|+0hYu*CI13>+PF2 z@@va3&6|87HWwGma74|$5zN@5=E=fC&li9RRIgHuKC(ZLCb5eW0Is4Ce1 zSaBR294L}JJUk&Por9K93eyrj~b^t7~`FI|FU6cH6=UC?cHSYR~L{MFPQ@)XOs}FXQ<`{HKgKU@s*V75)mSc5Z6Q;>mIJ8TH5=x%cjIrjIaLhMgaGDxDd0fC%8a)gGKwi!Zi zY;4t+U*N2v2F@-|jthk~5VuQ9Qyb4glzQuz?vUNyV2&CO4kr zNQvOQjOf!*VUuc2XFLISgs0*Me=T5@}wchS_`l+p;YTtnTF`C9fcwDB$J{c?wN zI(mEU&zw;)UBCD1#3tZI35gpwZk)jtVJDtHk8AAf=#aW#nn@d`%e(xeY&@1JEg_-q z%P%Y;U0q#3RM6XcYC+q%jQZ333>@;_U6qxG%*_1Ig)t*cx6r8#K>2~;8oo$`e0*_O zKeWZMOY?%57_b?d3ZPe;1yEnXaiO6|TtF9m7CSqwzO5hrMUZv-to1&cae8)FWY^R_ zKZ@H~xUv38$%frN&rtqG2M05tn>jiH;sS4Rd9KlhF23coGdDM{i)5FQl0tFAiV+YH zI1_VY>2~%P2@SJ~%0S*Tm{ng5gZf^Mx9ta5tpdi+ibH04R&r82K2#|=2iz`;@y-^Mi z!)CtjaYYGps66cKWiDS1AKjR#nR@G%WCe6_rM)ul;J-P?Qg<}n>z-8GQy`%-(Toqv9mgIWo%V4K47k@hDcOBHKN6%eW0 zJ38K{Nb9vfxyN3bL&kL?RnE(JvA6w0B6L9)XJ-h6M(iO%mR45B9+`N6_JZc0v9?Z# zkHmP)(Bw7fe|eE(x!9$}#l@487ahbRHs#wYYB0g@I{D1* zW?t3@2SMWpA=j_-aO}Eg{PRKV>nB!qs9Y<{%bs3d=-Uu%O?_sM9^H7y0-Nr`2O1~k za~2kZ@JSzh3n2(PIy$r~@qPP_vWFyBO>FvW?w0WV|8MI4zq^9wqpn1<7D!Gz7s-FN zqS+j@`J}iM+nKw?Cl5`-LQ<`;tIC!L=s71bv5h0_al4d2QnSdeBa@CNNsjdghPaZ` zmik?FS`o_fyscD9^g)08X|_nyMo!mj%u3vN(?6#6rTBz|yWeO=w#Btj&b*}@OJw+M zqQmen_nXaBCT~Y!>1QS`6))`cwqt}%pP$s?SqaU}aE@(viX>0^V=x=Q;@LlnQKz8a>uaosZw&4w$Q~lOl|H=YFEP6T)xB$ zotTE?-URor=?iHS9dk877Qpvv(-(S*&!0bM?mQvXQT*%a@dxkUAFi#Hx})=TA`}qr z%9SfT+VR9Ips*Jv7kqrm@~$dP2d=zH4)E0^O`@#<*s_&xTL;G)KX0RihL)kHAz*E1 zbMsWIo`Mhd5Co$9-XcfsR=I0fPrABxoi<}%zj`eFD1w8wb%%`T!U_EU6KIo-&Di+3 z_wu~c69abkTR=-__Zv3`L$3jQm4kikTu+vJU14=9DsStdRZ)TKiWL$5Ihq-p^z+rK zuWy$&F)=~wc6Gf|))inGKR!N=0fyrVUIaQ9925k%-}q8M@V1?c3k$d4+5-upa%1Fb zf}}`E3O3y`Hfm5&)W@~?I)&_?0hx(t$UJ-Y*r6uD&h(n=cY}p6H{QMLz#%6mvEXVC zfD*D-kW8DJ@^9Z(6b|K-yz=kCxovA;qty0&A{3#8>-$#$t|>Qd%6;&_dv)Op*(WG& z==tzo^D9I69OMEFK-PC$Ye|mt#d#|2G`f7i`ZcXk*HZA zz6@pQ7eBvC8h&5?$NBiJYjZv3vAI&K9{V@foI|lNe3s_tL63=7C|YOEj6hsA^;>Zb zI{}bGysE6cvSawuCsA>6r3gNE4-bglDPyt4UUL)h5P?z|Z&DbnKY6**^3NtA(~V=( zy~klx#`-!(OyL36KrlLl1gud zkLzgd`XI9F&%jFLG!w*MwUzkv<%{%0O^sC*1nCB`n#U4R+mtb*A8eA_Z2l9i4O9zelREJAYc&90b#--*+Fgt1XrGd$rKd+#f7?gZbWc;(nUzUbmwhPa?c~n2dG(uh zoC6l_YPv(T)V3xJzuuJU>^8G>%`TKao%{7v0^Bob;zj3s)gp8A@*w{S?%n(2=g+y> z*@yS;9WS(@ofU%Kig8++`^hK&=tMzIQPD9|J-X#|?6caMnu{;`s;a7x#mVE|i@F@L zb#g~ZM19AO3yOi!8!`qOycf^z-p8{evF*N>Qo+4nzob#Svk)p^KXAzg>I%Fr6dU+g z6G(7CHUj*8`0z(_mU?JI4F{7z!!!F%7+K}Z)7q-n9(dwcGtbSn01y}i1!Zd zSOP7-w1AJ#3J}NEU_%Im92{xz0%&MH4G&`;3LLthLPvp}MMg|zo7)-4t0!jP;W*P} zBi_U>H~TW@P{Ai#v;u=2_V(Xl3o;WhQexuc&skX&I1Q^2A)%rBFOMGwP{nSYpY1e@ z@=BETo_{RSM7v5Iwz_qca_RWBq>!vwrCw^ptS+g&RUftb+ecdBhJG#7`2R70hgf|w zf3CZ&)~Cr=?Go5*(`6IQ-keNBgA*VnhV2x%pMp=|ZfT;e72Z4;?V zX$)2@DxT)tHZwqvYZJ-Ey&yFdbuj5h6KxI|e*92iFqR%;9}AY}cKLGT!4rR|mnWDQ zosz*`o*g$IH*+48ZN0tv*zAPi2N8kAv+KSk+yD=;ITp0r67M1=+e%7?T|W;dLViUt zSztZ+Ar}`HEv>Lj_0gRgV)>aCC`0kIMKkoo#MMi6Lm|DT7etJqS6@}y_v(uXz+4Fwa zLuf#CfqU|$F=cMyOFBeEq$tQ0ot&n^7zG*Flaj0VVU%=t&kc<3QLl*L)yxRy%d=+# zNCIZVA8jO{#Q=2_ZvA>7!aV{9 z0(sT@F9;@V`Hu>6kBu6jyQW_V!eBvBk(;wK_v@&L2>x9X7+uO-`aCs?s5)&uy$a`1 zEex@uA|dwl-o22wqkDLy0(rHm^Jmgz|NGmab720{($aKzU4XcWi(gyrAl!rndNwGS zgr-~%wcEIVy_0~?wX?UsfB$}HLPX=OgL8c^T((O&95Hscqu#me;i^sd`fyF#b(lB_ z-LS(qE*i0Y*-wq!)2YYiWD8EHn=oIF*eJ4{$?Jc86+7$}tvXN8^X;#TQw$4lisqnc zU%R#;C@7nCazAM>vqSYHi-Efx92~F}-Q0>GWW(C%b*w0Zp>Al{x;D4AI*`x0>9Fzu zeTdn%D9^zz>5B#HBdBhFPkGxtE{v%8&=y#~N+ zA4^1ZG|hTdTn=SuXh;Z%oXFn2XP+DgImI8~mvB{2o;*QY)8q48Ubnhp=H9x6;QS(V z=~A_qqkqz-n$HXj45BPlUyt48U3qWU<#|x~`t<2)zvrJSe&Ra&|qLb`sFi4;(lE z5DP)J3#7EYT~b!I@BRCU`j}l>;oCAAZjYWHO}wFY+f40tW_I5F`{3b*1_r=tP)0c9 zJdMoE^cYSYKMveWMcjV5X};h}#x}ydVI48Q!1W>!)`lSIgFFvcV-1%hOPTJs`U1_d zsHhVGs<)#MA+>EQv7M8L#{Vy_yFMEY&BfJakNG)2u1mG8tU^iV_GH$N0(j_kt`q)^ z=kD`gW~&OOuBS{#k!w*&o==7Kx3GKfUSuMzfVz{ClIDI6Y}>qf=%S3m@?$ngT z3)`zJ3RfS!Z)NQ{I5_4}?8BV+dh+o>ZtkJQ+3!#yqn%wrD!yHu3;YAF>Dx1ZLPtwY zH9_d2yIa6`2F~jqeooyh;4aa;Ej4Ns&&0&-ey%&sZmzeHpj>i?*#L0lx3)CE_gGQ# zcG~p~c@v?KPaw@eUw;b5&)!XG%T#*I^q-Q=cB5@=&hKIo{YOv6cRIxF!or!(y8Rl5 zhLd&QV2$y^%@+_r3No}MQ$G|=F8m56Mn*)tNX7;7oOzKX1hl(!c28>Qx+LomFMi}n z{_xN`6?!8+F$j!r-o6FQzQGzjjZou5R4!akR#tUu>o3@zBBhg6#V!_e>y$L;bwejM z>9Zv^?-rPuRB8VTZ6*d(>^9^5P{c@g>~}uXlqLrb3q^7H>)rHhTQL$5G^9B?w=Sje zeM5tQprDJZD}ruJdwa^fiUKGMW0RBFHe3~(2$zew){-dP75eI5C?$wD@SBb@^a!}} z+S+enzfSvVXNW=nEnWHDi^aKd>yE@_A^iU$R6hv`B=zKdmy3_vB3YZ#=!@Xgxu&%9 z_V(H-uSo@Ww;S35U{QaJq7zwvu~RjYlJH@UC43VKxlsV-Ux!ux`Zdb?g^#8Cg@nf8 z@Mh8j)WP(cT3Aj;7WrHIeQ<`=>9-=IahnV%3YU)t{;kb<+TY-lb2V)|b^Q2vf5|Z( z3(ngUX`E1$A+*eBIsmMy@7cA>=~>L{=IuiwQckUHEze_O+!P{LAOVFS&1+2y_dj8l`l<{ zpxZ!Vx`52Hu&{&Fsb;=OY1r_nG0npj^Fvz@Z9!f-ZQlJyk0j)Xp$Go|JKZ2zbYH)6 z)gdm0&4~C=&^*MiY#4%dm!j3Z;_tuR7}VJ!TQUR=lRgqwvK=hHNpaV2XbR6TZzB#?35N3zToaY zU$@gwNJwaAcJ}CTI-mzXOiu7t_MUQ|671azlZ)Eoun zZrUixmiqu}BoY=hg+Jo0Vei|)gL~}r#gcRo*+djZpgQa0C4j&*f^aUJ4T3+)o4*_!%G%2%DA`KYz|dd}_Qy5clY~5~LM!7SHj7VvW3<*i> z;x(WUcU)_DcTgBe3(i+VXM7x{6?~Mj_}1->F|e&3S!@toOHzPk9`Ori8COL^`!KWPfT->qU) zKJNWjS+F_~{zyHtd|sU|t-z$)!N7@#J>sf&{N%~VkC=pZ(9$rfP^}|Jj`;a0p!+Ob zKPVPnOYV=>>`W!56QOQ+<-n!NH|&2&h0bmP_3D&M3FrP;TD5nVPyEJ|%Jh zzVm~@aG1Pjn$^MQK~=TF^>i#PGp}E#j%(1-(qeP76i(RE^vHn|!A-ckC~j|fSoYG6 zl;8NDWVCWnEO?MrGE`k6+e!bO;HWqfzps}y%9kFOm9Z0;t@^dhnwUFCg3i10Evg*i zjoa_Z_+J(p8Bw+GcJ2@9+6Xfd+qZhFose22^p=DMnC_cE;V99V4fonFQbvdB3NimUtLEitS#@uE_0hm1vU&0 zu0yPWS=faCfJj_Jsr%H>o135tf(>hvj-^wtk>SRWZY0p9Lf{~Tgj~FI31)(hw)U{| zi|7*e->K`K3pmWKC%RSz+}{O1(+-IMXF$657n-vcYXz3XflGSzM`*QVMD-3J&KbSM~-}GxuZ??>JR^ZfG9S` zr$q=`h4Fmxm;}2h(cifGk!OQQvTP)@>tZW zE%&wYqI59iQO$ouBAI$p5iTvC(HTIRMG(;dW?aP?jl`rRA81Hn;Ts5~yKzE-G*p0v z5fN0D#mHPpN&@>yA}5)s!$|{re>Xe($UAakyYvO9;A zvAmbGjFSpqD~sj#@Jh1IXomr(n#75ViFLnpwRd)&CaG>{qdw3cQBr#fZRv@3|H@@9 zU{pwwViCfni&Oj2`9N;~`}Ka_c_%=2PMYt z8M(Dtm{@pu`A1LUu-lIw9?zl)KtocbPQOX1>*?7oQz{yPZCgeKEO}mEiMGqWdv{;b zJAm_!4r8g5GVcZFAvaDURn`O9|Gv7qy0*4Uu;j4a@cEg)-lxeS`-xhF7a1|X_ez?) zj5pKe(aMP z$t5_i`PCV5$Nra+UcVihV<0Zhr&0ub_I;8QxxDOsv@F3f*2_`xNzL7lOp7P=O?u83 zY0=|_d^jyAamEQEX4SLs*!UK z4h|L+=A;g=W}|s_b@FOuEs&IuXb;LJaV@JY+v263fU; z!Xklf9rN2dQ{{aY!Cpz`SbzVMTF%Ys5FTti@@4ln=jZ(U%Jpqi)s!z^evY&*rXoCC z``9sNQV{nmTJ+}YnugILT&g5(gyi1)j3xcYDY8o_z~JWu;rJuC#(RGCcfJe9{N^fo zUJehXcYjg@>o@Ty&_!xtXec&r1t57~@?sjXqLF$iZa{Ye61;&U>r2c5VeCk&IxKd` z`Nhp9FdcCgDiM)#U{!~tm)GTp%P)0JFm;?oqtBBrRm{Xj`&9plq*BwONQ zV+nJd%QAQrGV~hlxUr5$v8kNSxGXdY#pF~kH+ExIlwb_)ML-sU9O972OU ze0+j}vd?&bt$&0u=go%=!->stdsKcTHHh!(vvF?F>+-XSMh6 z@dX#Z*k(}fZ?-tNLWqiy6Qa^Va}>Azh^KnUbxM&QRQHgM9G8bDJJMQJnp1!^v#OL?75FmVkC_lfT zY#^^{|4Z|j-81(|+N^}`X5D(3Cfh`0rvuh+sgHuVmndd+EkK!6WTOvc9%%KC&Pm}u z{a8ZrU*-8j;C~$I&D=L0(9ZDp&GIztU}|SNhChG&*d69nJ`Pykz`p+3_ky1I^l&9Q zaofPas_T1NF7FdtdllCEZ8|=Btp5mG_bb%yVntoQD9}M9wVC30DSfT zR587Oes#F}b}(g+`;R#Seev^`FUO7^z4LU3O<-QLsEo`!Vt3+6?||=0uyu=zA9sT? zy9w@<3FO`MpT!xx;^fRburT1Z>cNY!#dj+B3eivw4Gj^*_`^?8g0DKrwH{FrR9h<- z_Dnhj-H|2cXIk|4k^g#!oOMkWWV|EgnWE5!fld-rq=km|$2w<`Vcs)mw7;YztL%dCdF zR7!t)`|McrF5CG0I~*fVAC5#q@9^>}LCB>`1N{g+0fHe)#PvZOTP4488)03@D=1`O za|mBlFs-$1AST@jp`_bc)C#LqE)Jow$kz~19i{f|i?h=eQ=N|V2@p3mbRy_bQIYDv z;y(89B+r4TSdA?$29^3;yLbM_?qlRRYG}A?u>hf%FP}dnp~OTyztE2i#0KKxPg;|b zO|Fdu(bHpod!wH_ht>tc>j2>mZu6xneIs$Y-zm*HSbj`1$keGvWxk zLSMp>V<6p|OgpiRawFc3!pNel{&W5w$j zl@>C-WCWN3gnwiyx*eYH5x>m{dJ4{})xQQFbe)pZu9`9iEp`@(6?e|Tz^6#sVs zU@#30Sq^Phw|TLRr*p7@v-IhhZTFl)<<}CLmvltH)vHh^A+s1U3;6t{*G$aJf_oW= zE}YtceWf1dh>tsees*e#s}6@~aX_r>?p^f>)vyLC6tC3LkrBAdWJI<%S;b8C5}CQ> zOrg*vo_EiJQt1e28(=P(@W9`Y-=PJp29i~9o7mwLlV|roL+2={y5*{nB?E@&m9%2h z)AKJ>T>p^x&qT(w9W(P$)j!~~ID>};Sk*;V4xS%6du;;*e?qaV4tY-nzEVrqf66{+ zIf5P}5+x@qo2ZlX{CECco*T=eUm@2MqLPw4bu`sLJI*2o4!kJBZ7CPq3NmOy0RV5@ zfGmbU*eRpktBG4~R$Q@+u)ZSjfYMfYaiuLw@L%l$GG-pEib3T&wTG_PT9PT^)D)0& zs2EaWHS?%93rqqcf}|PGiuh$Wj+`|~1Yu-~P^r+n;m=p=32yNt3>YVmLoOdG2Wr=K`o{yGgO;B;_xH^R6iik<`_6e+M1f(&(F6V z<#Rr3W0RPiyyqnoPW&Nwi^@g6v-80TEL5+-s`aW2npYpNjIS72FXWZ5=;A05=Q3xm z1Ray&#O2U^!9F{Vcc~h5hvRj zigo7hUW{XS!NKY{3;=9qYbzj@&=;O9;aVP&tWCXEnIZOHw1O*AYqMOpV!KT-^rF^N z=BmXnULfr$$e#=Xe^>7tfm{t>Eixgg$#^@s`*^tVzuz`f041eY@7HPx|e)FE7sO{c{)cdpvBzbeP=l3!-6`kLF}LN` zZSId%@b)>$PqtG71W!y%VB6z_;xSJR$_DDo$N#s%j8H>&T!IgqR?QEp zV_*NR8wim#iW^kvhkgsfc@yKGrN7=oxx*?(z1B;Wv9Pu-vaAhjl3&uv5CgbV(xBFt zWYfSqdm{^k+%S$ozkT}_#~&HknUtKbu*V}9MFQgfk)(K(>h_A}Nl2n)YpY9$vcrJE zV;O)K>aqK7(y-#f1KYxC)T*NtR_7BcmTSAaJ@EVjsA^MFQ$V+7ek-%rsJG+EB{}bC zXk%F$4+z;t{I|Cso(LuVE$A-lB@|~My=_!fc=|=o^e$2YCi*06Rx|1F69l{;KahFF z7kH`Zxgcg$TbDC28LvGX{NJuxXjgmbXiHO56bk@_@ys_#HYsMHk=$+g_B4*78}u#G z800{`NM4D(je2+E^Enlt`i$-j<8n)J!^`y!D7LnKdkGJ(egC-(|uDW5>Lp z8)sY2o;?f24ySlCRS{z`zG#k*p)n8v=0}1Jj2C(gPNF^G#`RzK{}TlxEeDzaCorCq zJO2E#i~BB~%zgjy<}AN-s)6;OsouJE3wJAM`<%38#q&0SXM~1_A4I%fP3<)lSe0;O zmEO(!)xc#c%VIR`Lg%OYV}tu~=X47(fB;>G?K1tQwNc6Q_h9(ds#arJ67l22g&sxi~Uq%t8verr25u7WT}5*i0m z04w>!Zm{zhWRnE6VE=5TKSc>|vz~D_opZVENk|q(=!g7F(R+GDg)Lws?*D--7x=aaggRGB9(cmBjBm-Dz0(j<2 zc|7H`(HUP~`K`mE`yYh1S(FSu9=gV-IXWfCE#eyD7h9cKm(}1%v-zY_#u0;VoYDn_ zfz5@SH4f-Ngh06hD6c(NFgm*uC!(xe-PUH*9DERKY+-S6VxS^@z8I$f5C+5n`XkJw z_8dkXVkB(lI`_u@Pug?$O|>fW`hH=j&yS-h!g%Cs%06`F{kT_i;=I(MW8cL!@94Z& z4H1Zyad~Z@wA_>T2qt z@E}FLUUgRr#0g|n2^LmXvz8@{CKAKnP&x!j0miPbnuKb4|2%mm=BQZ8pL+g@Vddp4E)i&j< zzdhf;uSQ=pLyp1d`Ua%8P)zaUpVOB{@boAg&m1C;twef^UkfcjN%pssl4=X*o((Bm zv4q;LWQy1j-?nfmD=A?DF8}=KgXdEbc%+Iva&tJPhR_`&d%PNg%Uza7T)ufsQL|nD zQ)Z(V4N0v*N7=xj1_6*XVxP z2nlLTv*Q_e<>jn(z)F9CTVeTy1k1qak4+XB9)(PQ_MOJ zqC%|QUO>5>+P_o6hJ5^3$6sM0uq{1o=dN2xNvzDw2nTcRP$hwdJ_geC1pifU0f3NH zMGh@U%|CkRKgyB+%Vs=u#@NUxyYL7NTU>{`5j!UoG)6!*89Yo2fqiD?sB#)-RL;#? zskmrg^ztwT`RIu4{4MxjrG7g#JSxB}z$VO#rRv{@q~)*yodPl;9$-6_yH8g7(cjSA zAL-h_?=Nbxz;t4WQQ7gEHzj(JZ)Rk@dG6IIQbeEB7awbJ(m+6yIe2i(I&#edJmSQ5 z-GALVO+>d1?87Tv#Gfk3(kbD+`1Nj=iQ){YKPbuB+ldz~b9_0vw?VnHNu{` zu@fSDF7pY;In|K(dlPAj8m)hGfqzOg@z2BGTDrEmAq4llurNFE2Nq3O=thJDnwsi% zdL!Nbr?io4Ucc7Ub8^GgjFBUMChGm`3lgcwKf`J@a^|OOU|(%({b!bFNKZFFLyM(D ztp3(u+4R3UYyh%YIIWCy!9vELr_9~D?xb4{MtyVVMy0U-XmsTNJjLL~pQgWb^B)w* zweFv%(%s0pl$CM!?w21$10BuH`@zritPXAd8=C%GiF5mGkp8^}lK=_1M-yFo#Y$^_ zUUNHJUwf>PEjJ7vcr>iv>3XkDD1gO^k~T0eggfwLGFO;^TEQNr6Qh^c9Su1ba$7i) zHk@vozKT@Xk!$>6c@m8@N`#)_OBAo*<(i`Cfi!$vUEX(UB`k?R}Xn3gTW#8cIJS9ywU@~hlc}kdBe$#x%h}D_dkClpE z*E?vZKeqcj@&*bC9u6y%NfzpnFun+UL_Hmw#j%>MuGLS0B!8r9a5HJdn1z z3jVhIEd*udnI+0#d$OMO%pDG`V`ei9EWB{nR-8Iy^jF z)r2aES0GJMZyB=eG=3D?vjP~k!^L}4XSavV?`2zE?5voWl%}M(+Cs+3(7c*=^n7$H z@r3#6+S#d75`?1A`UTS@&dKfmbWLe(L`5;MZ8ZWWpJsCC_TL0)z-$Q>? zCvPAq*V(gi{(hWusQrjAzVBJ{X=0PN92r65alDn#snV(bJ@|bSK^)oUEtg@(&rIOV zpsjybbtgUE`QP{H?QIC=n>jFrhhq^6^?85G; zB#x8{c^%>?bD@$UWC$U1r4X6&N*qIGQiM2OGE}HYnKKhYROX^YWS$d>l#C7f*71Jh zb$$N$!E?^@>}T(L-}k!LT3bGR@D{h)2K*(6&QlM+lr;1u|KDFqT;;&mXh1Oy&me8r zoMdI|{=Chy@Wk|j(45!RWn{)dSH9a8m*?l}O?_0QKdiQK(WkDsTX@ljm05_BR}97> zVPj7KVXmyKfPiuB#oNKbJhv1NV&cQ~z6ztp)2zM<)a|UyEIYx30>)Jrhk4;swkY9KZe(ZEGmkA7C4s2y;erU#bHPRJ*80JBW{(9%;)Ae7{U{+@{bu{O~o%D z$c1KHl^*iDgybAHjiZC0UO|r%zUl4ywQuL!>p`vBRUg=Xf}7BWED;tG!qpBJE`YNG z*tDh@GzVmw!FNh8|NGdjRvF!&4K^p+5v07U%p=D@?5f+h>xLkDv3YsA6>4GWjy(VS zhDpml>$`8F9F2>UmX#Hzcz?>uKq3iY&sPBLr@Mj4{&=y8=jo46*|Ziv92r*8{TA)$ z)(>va!d69NHPLWr<(WL%cMm<4l^L>ASFwLz9v3GJttlpaHqda=_hvc~xY_!7PYL!s}~X&Z4dI+SWSdgk&UZNElUj zzpQp!$isDK-h-Yj!2#OW9{ohN^WKghfP%Zpe#hRQzd9R>tWFj3-fh%L%#* zHz@=q?F4NBm8P-A8^6Ec9SnAN!#Q-DImOWt7L=<%CU2PiEUmYbjCr2O7X;5Dv5?nc zoB#7!18DPd|M-FrU~IGqE_;H+cr?F@*5o}#yzz>ghv$YPscbh;tsG5@vOeyr235LP z+;)vb+ZMGAb%}r)6LkfvWbaO_;H9Uu#oxXykQ2_X=ZplMc<7+~mg`8bWfI>A?vj;} zL7Jkh(gY8JpB$17g54zH%{xNEJg|bl;H|rIWpskHei48bW(FE6GLbM@WIwY1t$W}O z08$quL3rehYd$0^JGJ1G=Bu!)6-+6hN{2XgFayD@ad$5<)M4I6yFIM?T2^7&CxiRM zxtTuWUBdnQtuA;cAg`2{8$W5j*RbQBotKvkY(K2#2}2_#Xh{ut~tW@c=jZN7oACT3>tl>_Ga zT{9k-TcpkO5$e`uef6q$FbAavy^i*%hXl@Gju-?aSW;OLjyxe=(h-PtN+)d)lxYLE z0DIUW&WKKXM+!yi?rXFolL|hgfH=8VeQeAkRYLou1u)r-7i;o7mR0*x|FW- z`pN-Hi3$2p!NQ}kHu|P5`1`EToKMv#U5<~0B9@3q!G7&%1gufG<_6C59~6{{c9^>D zobC&q9KLb-AAzJHgbbEGsgR)my2W1EAUWlGKj0)QbQuEiyiWUicfHLT8X8g6{m-ua zLBRqLRb4$q((G?!Qbk2`7(|oY_n#LUo+juuEUpb#i-g)$#E`2d-7_9l=HitD_xJ$? z`c$o=%{C4{5dXO1+h%Vs>YMBFvh_NuEy`bCMPr1Q??!S z4!U#}&WZrTXs!a)eD5XW1$U ziX;pq5Uw8fH)xm`8;{6P{QT}!TEzA?+1e25;&%! zaI1FG*3c+8vlnq_{{>sRg2UQ8m*)*2x+OV1ZR_e1Ocx`TQ@314)1h|k8&ZS1r!V_d z7bJoi)HYn@3DCWS1}11HqgKymeK2jOr*nE7BGq>6{s2BP%__RN`DW7C`iKtrYBG{x z;1DnfqlDYoN@hN)5wUSRwc_C5=&9$H$oL#_Vd`{ zPw8zi=e0j8*7e~-Dr`QQHTK`PsXrJU^Eh^o=z^djZHWhE6c)BwNMQRyiWWJ>Kt^i> zU3V3S4g0#t%JMnD&}waFjbQz*Y2+6#tO-V&Sln?(y?|bi7&fyoR@%c{ zLUR`;Y9kY1=T4oDM?$yk-sDtqt!Q*li@m#h=Zr`ALFSH#lV|yjzbT)H3HmHbu5*xr z%u~2eH^Fg_u|b>$GJalpAh-gtFK309;o_~Fca%~lcj(ZFJ5>sANGjXOE~0e~N<1uckHYnF z$dj?_L2BwAcd@puZt2Jhb_dZ3GeI+95eB z`7A)Dzm-dd*R)aJ%xs^S+pxF4pWoU1uB}_Ph_p3aUC(?XnM!l`%Fc$0RVS<=gjsFF za}N+jn@Sn(2mBgz=C9n6b*U|O=->c6RLFbYMu2rr@~Ik}WzMu$U(i_LPKV)}N)xOS zlRXcui8F{!v$L}|d_+C>^7E@@cmBxpo1UD+dTigAi|x}K&F~h|GF^2oYpT_-Hwuhe zWSi|n$)#*0KFpM8EYXa2Q+~5ab5&ay&%FYnMx?F9BIuym`SWQ{#-Ys#ic%_@flKxd z6+qqO>O?Bj_L<$|a@u&uK`e32Y_CY2_cdO8e>y@ z4VDOz1S$u1Wm+YRMq^zn+3Xb;Cv6S^FHlpOzG4f??N4rpOUxpcml=wQ7gGnlC%CK3 z^Gl<9)nnMyJi{64vUtP)T-$n=Kwy)hoe@y)I`=`-#TPE?FKxER4M93PJ~8p#K9PM+ zr|6-=+E>(_ed4T~-xV%eVI~&WY0o_W(jDI2iSE}n$66X$Ms&cFiujp6VOZ0sgE^|{ z^MWgv;O~~1Z(bnBpv#|XdQxuRhww#JB5j1=!Isu(pKEjx?mOf?a&o3;Sa_2 zWs=Sii(`C7H8dO5GD#h>W_yV@ZPca+VK6`#!c$D$#xqX29sU-%Ye8>Vvn^?D2G@X+ z*)=%m|6~3Fwpk`8s4QbP?#g^du@G+25@^!EYi-(6?K~2wn}G&M()NAKx13_w+{0L_ z!A~eL$(>gbm3Qz!36kNlUQgAGZzJ~0$(W=L@CG2wyZo8HxF@GKdz-bTk#gwnaZi6Q zYwK%cG9m|lrfM2V3K%ZiJ31cX*j#hK^hP7zVXaI5uV3<2JG}TlZ5PIhSB;)MTQ*ej zLb_g0p_{3A?)YIt^ZX^K(~2rAv463+V%ywE`b*4bhp->8`YAW4Zdk#u2r|@H2oQWI zwl@5%D`xk`%P7OA%75McTe9UBWAolaH5%0Z^sW-M6uFN?t*5uesdsI$44URaosK`Vfsz{{ zlc8a(qc5~hsUWmjk@W|ZoJ0t%cJa3?yuA~~WQNY)cx0BuPUjoODxb2$OHVtMMaYy9 zNmuB$p?6#}Sp@lTOQS$=Bv@3?I=AUNRVlwuW^uc)7rWrsKtV$W*85dQ5t!sWDq-$z zw49J|lF(3-LkA+dwyrL_f4Y^o$(i_-vc^F=^JG<`Wj?3vEMA=uZ_Y_VPmS0u>yPF# z-yTGt+^fgLZ5_rstuwqLV93yS>VWU{hwGHqS*ERrsO!<6lycsA-;s`I>1HhN=Ojf6 z;xt|i(j`eot8_n1>XE}R5E(?-p4%2{?i|W84qd7-c$}WN)zX_raKgm-@2pQS=ac?E zfgQqhmO2mhZ@T0?TD$fUixAt@i4`#Br{k_yzd(eU!;dPPXvyFp_@)UQEWjg~dgYuEDPb(HbwV%Z8~m-#J_I@28k2 zpClwrKzYGHM#);})LB|smY%p&ChurvrIPV(1?`DV*dtsDQ|}bs)EE(MqT8jXIgnhs z_BgMYZW5n=_rr7h_*A$xqn<%8)VlMrqm`46+-$M@40>T;7x+KpTv{uTf8SzhJa>8Ghb;bDU|Q?{yvAS4@L7e(Is&j#?++6tUaIz z|B+v`4KqMY_oM}`vbU$Qxc*#peC7)ibVl;%9XSyd;qK$943Ys8OYc^oRB~}Cd?l9| zvUauo9^bQjI?phtY3S)GLeVw5_;1<+6RHt@wFj)^BNAbJ)jN=zTtC+rUll(6mG)k_ zZ;Q}B>t;xEKAdQh3lDpZ3KtmJ+esoi>=PIN{y5k$Xzux*ikKW5yL=8+q5v%_=qEza z5F$#S^gy2qu2`;-shQb~+hJc{-yyFtW|b!JfqCUELmTv|!&|i<9)3wppaz_lq(!Bx zoNYxNE+h`7G&I|QCv0cWPS40FD=A^TC_BZR#xVbUPj75ad=|$qsxd975<-O+RfN5g z@Iq^JzKid9IUR2X4hkAbPT6O*T zTl-=kum%>~E_oN2ut(IW{=mOCfn+*ti2r(Eo|oLYZ+~eiGu@8KiDzwI+iUC}|5tsY zA{*b}B`18FvY+{)vE07y<3(0_Eo%Wf^JJ*|H81-p|BjSQiba;ySJCAn50ip8oBGm8 zW;)#Vf{)bzZxW5?Z6n%uA&WFBbu&170w0c)^Nq&mj%ef?!hhQx_e=*XEIzJ<6CH|7CX`qv;)J9NR z_kfa_lfzoJ{99tAjPk^FoY?5nK1r6+FB}%r5ZYM9al9z7z?qJ4(g__P-Ha15hnV0J zRbt|3Zk{w>0Kju_+_ijwGO{7h0$NjGj^JSyUZl)}nt`kUqWAmom!2x6yENThh(2w> zU=e_JNQ%_56t+c2Me(j)+D`v4BLjr1jAzgO+;yss-gTk4M}Al9@Nn|>*Flr|753|{ zIu`ZK_*9wy&{hd>Fho`t6cFrX`1raZopRT}CT#&@WoN{2A@)p46jo1#*Eg${JC79} zTsJmg)q3HlvNKlGh=mKoR!W4|Hg$9|JM1x5 zX2Jb)w5`<&3}o=6v9o$q{zm--TNv@G`Iw=xPVeT?Ek}i|df!I{8*YYY?edXhYk1WI zd~#!If?T~}3>zKLk*nOG@fXQ|1PK8gtRK6wKA!qf9oPN#DAvL0zvDZ1gp)@I&)hfU zzbER26z&oI%7isg433m}KC;j_$J2jAE*r<+8lZI9msE#K1y!IEb*#dm#L`Gldy;rk zI`Z-6!v`S`N1NgT6H|4!Re8D zQ$DqRlm^cakj<_ynzEk{(kDKv6e1QA`Ra(XG)awA`{}d%hIRXJ{WR0(+iKLHUAAg% zD!*Aop8~0a5VgeY{Mepoz9hssi*rI{d(jRqEh&LZ3$(mwjDJCoQ|qzhX3Vp;5`D#a zqF&CN;bhFk$qCUqS0fuMYqvuw49#crAF#>+_|+Laa+Je{ za0;FSSmjXLuKt{DUOWQSwsxzc-we70-s88xQ|||DC0ae?O_}(p1osiYWf*<1Nqs@D z|KdeaadAiiF1Rla!<3uIK$He?{I=Ssnw(`qx!!utSSb#T)X<|? zyrX6?#8gi~ZS7Ki7LcEL{^==pHKnhnTkW3K&8@a?p_>KB z?0-F})7l^Vc$=c8(c{{I9R!xQ=wSYU`d$%a2j9i%l_|X|v2rA=+yEBr=2BAA*2!%0W71! z7gV2_w46X?GwOW=mKA7CG_9LA_ndN+twp4`%7fYf@w2)Q zAUp#Z%+xXOBRJ&<3YfgSH#P^R7JdNw3D0JzGM|Ny7xG|iW+08?fU#wgw!k~6FMwDr z`20pO1H=S)m7!sbZboxc6N}>H@r-`N_m|(F_BgQbvcp#IjXE~F^HHZgnk%}Ahxv~< zlZAz$ZO_iidTCV%{0=x5Qb(=6KXu3Vx(5n6zV#ho9(3ijmg>HWmgfh6z6@l4!B2pQ zbu%v7*$sOS+8jtrO$9yC7InXc1*DPOvaYH!(?Cfg zixLx>=_85sx2-J}D+rJdfAONm4v3NZbwnYC{U78V8?Z+-p%)ePJb)Q^}EB;Zhqcv|6D<`q@$XA%r zT_Svt~IL+fo21tGA zSH|qV72p3o?Sx}wq8__Uz9mm_Z^8=4@omvZUI(zaLQ4yu5SV6bypY=Dxj1b+G;z@n zZWK~?_bDn~fu9g0-=mA?R_Ef-zz3uZ$nuI+U&|8&RlB7hDK&|Nc~pCDqh$Mw&Em{AD4fOgFc3XZ!|T- zzPh@*pX%p7TZ)|rghl5Mb4y4{7>-f9=7253sKAvAj^akAPxC1FK&sF4)Uif9ZneDT zY+!-H-Wq->$dCJ_AKjaMw>!OM0e>o$#3S#8 zEd>4HB}6$UQyPx-1(1IGlp9t%=zxb4DIn;T&wj6x*IVPkLZv$PO<}$7<9aA z(%NSh1QMfggpM>P$W7~UVRh>p_Rk?tpUQKJ(%0z6uysmmNrS5}dHUIpPV~fJA4FsC z)m~?&uf=?Z0I6{-x?vGmhvKi1tvE^VBeDdXXMn~B2y^(RFX-;2rwe3v-noDO*_yE; zgrM^;1@AYl-UM?ol4k9TzUdJ;xxV&xnncc{GPp`TwI?i*A?!>;8=R+q<+&J{^p7C8RhnzT@P85yRv^=^Lt;09`d-;%<~|% z(+vCo&MUmxI4T$G8PiT%t-tDnp#)<(;k9j*`F+T9apc3)2~vygcLcz?dm+1zMZzbV zSozdEe}JN=qtj_8$sP{UC7Oc_l{rSAq06j4M-%ZP(!HC4eFK^#Fib*c(8EKj^*b!d zAvSf8Vk%@A-4_S$BbumKLMd$!l>!IEtg0Etl5Am|QI%Gw=!j)cVK z*cFCPc&B}4CnUFH46%H#{OlVN8DQ~R?=rqS?b7Cl&wh=IC0BoiXuO~?SN`z@9rj?We2SYW-!u-LLKIiA30=;mbu`1NRLJZS zbtjNH2Ja3oXRjPsg<4Of*M^<8R}oBB@~oFX-gQbv&NTEUa~(EuLUJe0ZyT?iGsq#` zxqEjj_u(KaIgoZEgk0KDpFkCdZrxBBkWf_Aa}ozDO_r3SC_w(%M{LXoue~#DQTbpIA@9_Ct*L%9Y5qdgmbkuCr1OkChL;bKm zfk0kPAdtNxlH(`AtWnbVw#h}^^df<<`8DakWN~{o?;sGk2^xo$4LlR4KKq#;@A@r0 zLlJU?+nRX8`X%!r<_-zNw}P4KInQWwcx}U(S@o{2(eKW@weNKu{q93ogLa#!GH^M1 z?X%-JGAbxkWU!rFzafFTiM@vT7=w5!trxLMak^yjUZ~5(h~c2&`L)T>#Fd%SvEEua zJhzr2{-ap0;@a$%!`@Y+ch9Y_{hsW~7hr0cXwN#}+~;IvRcKQm`ytnO{a06&|I7f{ z9c_E!ki6m2UNZLCZDfSf)HZ@aRMVLQ>lt}@?_R%t{pr)nub-2{T`ohfBeJuzz0N(& z%bS}1{DO{-j)8$e-!wt&RNoU@nk1e4?CkOJap{1yaueZ&*;xk%hdW)nx}Ds=2-8wg z@EGXpGqSQu=RezR+4cD3zJ2>1KYl#coRpiBqoJ<;+-2}p7~PJ_-wS5GoF5L~r=_`c z+l{>%ha)cdbu(d{{rK_rs2vBa+jkr||KsOR+d`Aj(9rzyIqG`Jr^Ut19_bVm6xXj` zUvl3W7Z+F9GgV~QV$GG5lq4@Ne@I1T19{N9*i-Vo0DY6cFg!DNng1C zg!DC+P_?=<{)yN8Xltw9mMu353JN+qjqsK@68`@>k^xN$z5Ry{)m)nF{JF58nW^aa z>&N9S3NIfY$u}>ZJ5aUUG(60&G9G7av3t*+Jz`>A?d>b0_nhm5f6g_EaVh$j7Zn}X z)&1^KwUX4q`|H;)2{vt^48`9Qx-Z^$?S09;` z42e)(3k^MDVPWCpv-0e0R|YQ+vF82zxy7pW-`6>0*_c^Cp+?O!d{Rd$J9_Vx6qdTv3$MGSDXqy?mdkrTA-{&$rp5&uh$1apjKqRga!0 zwr{U<_$+?*5D^hc5H@{sajaQDH`&s3xMb(gBey5_Nckw|J=aN;o?Tn{Zf0U)a^eK1 zg0H8Ijm_fpXRQR$OD-GCAiJAv$H*u38fwQabJZG z{{HRTw^yxA;j^f(3wqqAB zUKH-=>+9?2FkoW3859&$S*Z}BXJs`wrLBAP=D{|`P7B4WzkeS%_YrR<%1k{!H%G5xXlQ7{a=CiIMH<`h z^XJb8&i58Pd2%%}vZm2!G`U&T_x8}x(A2kYxw*NNkrood)v7=Da6$PG5EQ{e+vEni}T~0ea%Y+}sH_Z#_Nw_#^&i`lmulE3J#Ry}sJ`;e({2 z;wf%=95Z)_ub*E_;z2%oA}=rR(W6JHHg4Q1I#arttNJn7r65%bz1=xC&jthpI64Zc zj1SdD?q|$;8WcoE9U6P{=H-?E;FMaVsO;xqo;$XEt?$5=fC>)ojre?vt zEhr{VPQnY9Lm8+;i9GcwLc4C}xTIUTxplmIC(Or3(Sj2_U1$X+IoA1mnVfo>uth9Lp9C^OVduhtkpf8AbK)Z zuD%I3YM+7v=c%+SS2iYZ-M)QdbX02ocuzy`@jIh3M|orF>guvHGD4*JV{z#o6U2AW zH3(W@+vgn&u7CF~Lr*Y0J$?RT=eu{bYe%@(t~OajS6ePEEm^jstPs<%FsDwNne9-? zE-bv@J`uB%ZT0=W;p_PEyK3#HSV-&V7aj~M=AzqDdYYQFr~R$s zMNbMz>UwxAj*n-TlpIJNnwZ#4Pvlhezm{cbHtZrD>l371(BmX^kAscv*WeYdzS4^u zd7ip|J8|gHA?nazddA3FaSEo$S{%qcbxnIaQ_G1mHwPu9m&@~Gg*R~7=6?QsR$Lrk zD#Cub`lXHtYrRT(`^4qTmtXq`wVPkbsi^R?wpN*Al}B}`GbSX)UNv^zEt()JDS7F8 zUxn8>t=L|%#wp5Km4c^FZ>_3lwKJ&j#@H_y)%|m?bM_K9ky_K~gu4tXZZe;%tF0E~ zYHDh*6cn7CyxrVxSvFl23H|i^!k0HuI~11&E`P6D?|b`JVBfyk`4+{uoet;Ei#}`a zF%dTM@+!5tT3P9qUS3tj#lycW8@`>D^<@iFHw$+18Rxd{?&oP~JSY+w8T=w5O;`*9 z0(K}t*!sjYpT+5>>yFsbRaO0MZ5J%8)W4m!u^H}sWVXKC5+EZZV`ON!b?erqzSeJ_ zN{XI5xqI*4#*G_MSMbGuK&VvKvygsjcTmrj-Me;C@z^*zmT_Ydog#C=63WResC)b7 zO~JhA@;Qxf+o}_67lQS7UJJM-EEAerJyoMN+DLA&g^f<>!%=PRlz=s_A+Lgd`}%d! z?i=aD&WkD{q4al?lizTPzLb`gU0GQ%HZ)92@gIHj_U){BeSl6Z>V}2)#Lgi)tC)_? z&hy;`%Hk(Smz5mF_I7r5s@>p}_x@pTY;26iVPevKSMscd=OE?gE5{8DfA;x&*Em=B z`0;sb>sN2z+TpB|lu!A5lcc(Q?AWnEY!6AvXMQX0Ige=R=$3!|^7iuj(A6a?Bg3|1 zN7d3$abZ}gdO%blP*0K zU&_LtY7%tXEYWYN&et^XtMcN8*2CtaU3GQiqfPfbHBX=Bi&8yZ)I;;4<|qPkCKsqn{wNl`*0efmpUT3U0nPHCocL6&~_7N#pT`^3djcisda-g!X1 zo?0orZKmk+j~{Nwk2C43%vu-3eUT+yOY1`BZWh(TeAa!n)V_~vvSuEWWd`q|Q(`hT zF*&TNIyy3PK{}Cchg3&bS3-RJE&+klRof)k9zA;W?Zx*>V1c570`AgVM{zJbJUpoS zzkmPs_xE>`smRaIfBN)kVIjlh8H-Za%qv&a&YXFGejOF1ZDEm&%KvKRncKws3ZF$3 zSaDWbl!mpnwWm)d&~jEIp7odw&o8U0s+t(t*=6PDi>0(IE-tDOtr*H)yxVqtU*F)`<+yOMy0+S=L|E?mG`nVz1eK752d!+_G| z;>FzTY$7eK#Qyz_EiG@?lEge`t-ZayuXDGReqJW0`zIW4;{=seH5{m|>zhcz{?R#eD`s-R6YG&jGo0~Rg(0|*~J zdK7n}#$2o3&-+z-dpjU@U0ry+4bDwULSmp4D8Q`P39STRY;e{yHdlWjrZFcc=h`*e zT{kt-^roh!UcGt+gqLyV=11NeZ?R0f@ zBfA5TUL%-|t4wYOMmhpucE>5vf+5CD+k|}J@)qF zOG}MH$KXlZGwsj2zfJwUy(vKHIa zxHOv9hwGWp^yK9$CMFy{9JW7q&TqJm!8Xt9v7Z`oD>E}H>+u|eZQBw+GZcK{g+x3f zIOIHCe*gMOs{br5Nz@xk1J>7mPIm1nm);lLP*A|n7qC#F=S0_)f2zOu+&whK!opd= z>3+{IQ7A7b9FyC#be;RkdtSY=u8-ZBYn;FKGe6+>KsAM&2S}1w`OGWdl09A8y?Pa}cz&!Uveuf5Dh$wvXHHRDHmfUAuN^p73&W zbF;C@ct=9+`|X;sQGkSJmVN8<0s{$(ii&=Izo)0q*Sm3Xah*DKDx|fe;}$iq0cWtE z-Cow+yLT_WEhyNF3T9V-q63Y18y8pAKyDoS^_%=dvxibBm2FMvVd28JU=sjpxQHm7sJtferKMi}{t60b&H)*{d`XD61*Y9HC$oP) z6BARBL;Hg}ced9b=q`TttZ8fDJJmyn4~N{lXQrx3mE7ImubZYfH#axxD1pW^>nXSE zrh|>mYwJS(u7h3v8b<($XlQ6kocqr^I?Bk&$<6m45^i}m2TJVe<<(}7s<JPlo9E`N=1{PWK+f(GLz<)jr9P>gwtm8ymZ_#Xm7VKI}%TT$eI`pGRgq z>w->OU!TR$-sTiV9L5Z_`~0Xj{)%RsC~gIc3cr8=3dz0OxAV+O9%p21xTiMPd$8-e zR=gp)?7Ns(cEl=im9*CWey$z5GPqyz@=2C*Q`;G%!T{ojhiy>8fRV<>Z{NKuvu~er z`8It?b8%O3HOGb7xWiA_`6($W84lQ)3U}h}*=3;)@P!m84{1oT5wBZ5hx?@ulAs(7k@+M*s5*>1AbQg@yZY(HrmU z3HmCesHRIwN*Wp&^*z6E8ff$N+1y8uYAcPueEH(q?<;adm<0e0&1qma7QMMBnpN{N zb@V+0&_-;R!s7cT6Z#f`AHu`w_*I%8*tqH!?CL$B%C99O_!xH?)N^(kojlnz=)%s<9&Y^HXYstEV%78K(jlz?55B&>dU~y(>k;&tM~}({ ztoeeFf~2XCzQ~s??uCbkH#aw<`Cv`r%H#gkB}NVGVPawF#K)o7Vz+~N9y_M|;OQnb zW1KLsDg=$WX+$vkCPi#=ucX4q6v;orafi}=n3eZGP+|ZrdmErWc z?8EEV8IK=d_YBy{z+V0QIgWJt!-o&(?)V6B9kI$K@+P6jrDtlER#r}xx`z77eLVX4 z5lgCmtT7uaZPhrTAt3#r=-XH5@iKXJ4D!5&S#$8km-gG<*FA>-9{c(e;XuUmydX zMrvvs*Vc7KbNm!NrrA}90pIA0r9xUEjW9RH_OdK&PP>1fR}Q3bXuV=oCft?A$ZKcu z`Ojsz?oM~5uYsboMHv|xxvwr=*s~`WKp8wC{4nqj-|E*_Q{0=}+4t;O{i%gLsH3AJ zo-gsv@L7SR5{3ORa#m)*zP9d=i)-YK0YxKp?k5k7x)DCXSZ1F);&g^-rm;RcK9z1 zUb%O`87#d;^5S2-aQMR!EcPI!nWfp&Q+(C_8nL}<;(P3hKnrChB!1LvQ7peWwpqpg z+s9{NVPW$<$9Sy^J=2~PSkQ_lL}sbS*P9Qs=(aIotLJ+83LIiySYxyYoi8f1sZO@U%3}|^aa@j$ zS8evr%LLO>owl@8nwH5t1F(FE_;YdbCap%Ka=LPs)pxx&@lT5er)!WBs@;ASP1T#-aK>el#t=M6OSkqDvcVGl{b#)DmdPtW> zkIk2R>{50Y7ZG`6lCIN3Pk3{@z(FSIu}S=JL*Zc@>QGV~Z;54`u{CcD@rKcP*6zt9 zdKEGWy5hIMZ)aQ_K;E0pp20c~OG%`E@9=0RdI%>3sE;;Wq|AJw2wn3$1IZh2k9T?LAS> z@9g18rZY=Z=jYuX%W_$jB3gTHa^K21p}bgd{_S{0H5gxwo_dF{ntJ(I%p5>lD z6;CLZjqO(!=C3#6zP{?ASNLT`ID^Xcg1h^|;-a&oqYBZ~%yqeXS%!BQo}dM3mYRx;a&UG&uy0?r zj+O))-MWg7uC7+|!ZpZ6CFehTPxqGMBmswm4<7t4#<+j)UeATSM5DEV>dVm5PZvG1 zsQ9%dV2L7BrMm3f)&j>5LM)Lxm(#r_-tQC>%ipOLFBreG)OAXtwX-wKFEKg!yuJO) zi(@k3SGP_D-?|MQ(EnS4^fllmg_MA~;d+$a@^ZPUw1CZ5pd+|_c|&)fG3E4IyMzzs z0jgucev$?EwCGiIQxxKMY}zN9aMs3#f4k}*0$ihFxFnn1A$I*1b{CrA@YtAEW=ToO zmjo+DCMH}6#os@U*KhY*||1#)0tNxIpylq^aSvCver52_#m}?exvLh!m2)S3o(sqmlo|;MxB1Z~ zVj7?tdJbBqZDahA!-t1QMjn|Kz4`FrSx(MwQBmNX;YTHXvAtK{aU=-sQAknj-^gY3 z=tS*^>wa-@6XAGJ>JumKqLscosNv5bEG&%105UAi-i*=!-m9gh1>#-!@yML4w9h1U zzy)SY+yF+#*!cL){{HV{V`^FL5RWyrw6O1IV|uY7{%_dztRYI(@891863k0)-ehtY zqX`=zir+a3xCTg$juoU7os`rx;_Bn=U0`1ReB4c@+KHGJ7Dfeu610U}*i>8jwoA2qxPo50TdM-)RDJv^4ef#x?I_~h{l>fsU2N$KfIyy{DNnci5 zOtfOXlF?f3EJJm4Dn`kIEVwEdf_7|SB_lJY8 z+75!~7d!O?-)QGv9+s(A=08=?Vq&ciT^d_HRE6gPE8fgahQr4iD)NK;{FdHcb{}?2 zUthW1yX%{qH~Tbm22?7|6|a}Nn?KC+l;c!r2<|veanbP63y<%3Z?^5*_uY91iiGP| zKX$|Ab)541|EUoasrzpnW6KI!{R3JvZr!?tUiz`S+j1upD{FUaE4QSssp%aco6g#o zHsVZ-j5E;?S7tsL@y8`3XgvxUAhNKqKx;wW!Xm|byL*?7Eh@}!8-HY^C0hA*NRB{o zrPOL!V^BGveU;D!OsIJ!SZb9Zsd~P0U-e<0iN#Q2xr4jfpV>M6qRUte3S`oPh)4lzj_1W4bNs3q9`zkVNHY6)1IU-;x~ zW@eUeQaA+kW8gt@w*x;ar7K?teTI3Frbj~4Cr)^eHFI22+koF@Z!Gr-t^=QkUjpb1 zT=Ic~20u?dR63-w45 z3dXNrHhcmm{BWOmc&u8I<$*5!zE>=QUx1qP+W{+KNjq2kWKpTBuI7=Jehy7o*`0Li zCT=(+9yA9zIbZi*b}4>m?CqDD4)j}cIe#v*wX_U8x%I|amFGXQ0JqP^V44wSnF2o9&#Orl7yRR>0cXzJ0sMz%=QMBkOq2O1!fL_5d$~I!H`|cmSf2 znwol`-}j~^Ly%G+y(>JQxw?zE)Jt87e*RP{G-fy$VC+1fy z+-0;Tgqw_ty{9LuZuNsgH@t?R-j=+Fj>yA#!_L`Rao@feprl24 z1}CW`raMU!@X8qJHMdM}_@C6sL@h4=5;1fB(+K%>4Ai z7uDcpV{|I(Li9)ruaT|7Bk*N`u(jX;M|)cOo>vM>=A<1FrC@G!(%U7jT=?!mN5G@V zytYnQ*D z0Fg zWjdg$p2~4O-Fc*$=|-p6lKYBIeac$}>WB~pSG)9mkJB=QmUA2Xb$CTnsnB&HYoQFI zgn*;Uaejy20&4?p9}O0}4vYmoEOtIc&g*@D|2oJuwiHA){8m6Gk-PLD-S+p!LG4$l zd9SEmexh`dEIO@douk@!!qX7Z1s_xrdKCzExT!GH1_uX;Y4DDr__=v_kakN#Vn4|T z+Wo$#r>DQa3UZpoJ-&7$BcWQhA))c)z${XIx0eCm)J8Z!)Bm-_K0(Haj8Tt^7i+R6 z+m++4`(zJsfi>K9AZVH))WEy&gf#x%>9v-2+OKKzr(0fAI zw{Pz$aZU*e0#*w$Fv9;77447HymaBhL7#=ZvL^%+!PaAR>cJ1mC4$~avYqd*{Py*$ z5Hs~|=80VH|JWvs{8;ejsrPFFsQr`c3r?}fa^tnkK-S5UqBe@}aK zznz(J2~u*I>=fB*7n7Eib%By5kQ{X3&k|#~(V6~aaU@^#SNg!$D>lVu6V9JJG}oLK zj;Ztih>_h9b%AcDVg0F-Ywk-Q$wg8-n4;gDbO^tCm3qsTl%&27A9gZRBgQeq3~3C8 zX-o`*;Te&cKBUW{YY~lk9l-=ww-WiE%*>I&nk$G3xMQEVP3YtJj%Tfv9P|1QDHg5u zsiGDXsH)}B9aCRGT_MDv0V?L$**y)%^%>=*C(^3`bb4L|P&W}~A#Qm1PQPz6`R5HB z9|HML<_i6_FKDUKOG^*#+7O-L(#h*QFE=~dt-L6*Z^bFN(8bw_*BfhvC99$04Avn<FdA0v)>L*91u#%Y3&W~R=!uALR`koYz6Tg z?hu`1xrwh|y}x$k00#8+^`Vx&JN?O~?9(lwUwo1q_g*>DZhu=_nB}Yr3!#OavvMQ< zgr(u2YiV}~G6%zIP%uW9w!O973(Kou0%ndVt;IPbJRZy9-ZaS0LBa+B$<$5*ry376z4;FNB<&{1#Wfj)_DR zmDIf`|FR;ob|f3p^uVWtit08B_eh3%s3t4;u|2TzZquG*(-xfXY428Rim+RF`W2TO zCNroN^z6vP_o1vpEUe$|DVGdOFG{tsu1@dxwT94ys)qAnt-S&ic8abOz&GE&2LK{Z zeL)6FRZT75qT=C@u=o#@Wu;~m3CZOrMVxZ3Ir}8a9|}ujJ;5UaUOvtB^T!Wxvwr`T zb0Mu;6jxtb?tKiHkfsMhf;u%ZG!#q^2hv*n4uDCD-@xBYGDeNfMe-G|fdzva&YvNE|%49YCLrt;}owOo*PP zB|x>?#fwEK<##U|?b<4LG3`QB7*6ob8&xNgV6Uji#naK=UgSE`Ff%iQWdoj%^n=vB zlcuH-AE-j2g^JXJN3jhBKRh(dUGgO<<1lzZaH(l%xa7T=@lpQ2QGYGd^upJBzRWMf zW&!h3a2t>IK5ZnlwcLmN@%=rN{_j#f$@bAzjnaN*x_hW5_o`G|^gc%B>?k4WpLM10 zXJEcTESvI1lFH-br%wnm8RuI(%CLVLv1|PoHpK15q2E7^=FLQL_ZlqsVh=E=r1<}u zc%Xd`TkarxbHXYAobte@VY{|WP)clE_1J)tOXiw0P=2VxUL9hgj*i6(cM%mq2noA( zjpDWgVWN3yMrp{x>JaW56+*o~qDDE!abFeArWztgr2Bb9kQwGXUx><~Loh<(*lxN9 z?x72$P2!*-d$$e!Ac4AB_VA*__~hhi3g?rMyX$Qr$)29r`#^lV%N-b#fkHXU$yA%K z+)PMVM8Fd}2_@uo1jOC$z=%Lra|eQrlA0v0(;DYYB8E4Ix(+G%H<1maO2K4d0(J_U&(k0$$5`yLA+V}z^r zI6t3@lhfVJ?WnqXH*6>^u?m!{HH0TAQ%H=CH)ebiW+tLT3)7!{zI~JgQb|mdla_{P zkVBIt^0VCM)u&H8?g1Nb6SI(zl>GeEfhkIL2Rplyz5U*O`y#@__1sNP5J*j~fE&f~ z*UHM|*RSJl-T)-NzUArZry;kxy3S$I0>d(>kT@n@0ESAhVr6B8b`M36mWprZ&P-s^ z*RMlXM}Q9Z&Sm&kc(Fi=?|Z~V zKYR4UOkTOEtALv_^W%qs<&ms3{Otx=I6tO6=RIRmcR}z%!7c2k^!pt zp`#V`<1#4)~JlA%Xtb8j0R0fI$vO%v{;zbnz z)M;YC_+fSIQCMWQ|YkxT$-`MZ9;4npw^&su|FXHT{daEu0KP(;P=X+)&q13 zD_DCh5Qmd0>HrCneWnjO{rcH;%1->w{Kxxm6lSyC4@ZST+=LmA>?JyT1>cV7jeIKP z;%>`>K6}OV4C}OXbShyE;m=uVRk4Sl-(0#Rxm>&L`}bmR)sWYc5)x25%747qYS^r@ zN2JdYIWs0-*^tZ8slHAmtwf zm@TBlgiQ#05)uV0Gr+LSCmrwYfm>tm`~_Nfrj*S7{23J<&WgC`j6_rSC-b_ahIV;O@ffbguf%!eevpT+;2^ zo&zt*aADJjw@jn2_k8{=FC*i4{(SS9B05@H1NzQ#OR-jHZ*So^#i&0WFm3pwhJ`@5 z>&!DvpB8l6hDwBs1oFihrD|4kKIz4a7svoZu|6TQYg9{^C0Cdx&@Qb=jF~V`X-HXg zaZCsD-X*oP7vXGla-8{4(Rp~%p#MPUhfxZpPe5P{%UUS%2JOGuU6p@-xXCICneP7~ z$IUGD38W$I&=u0E#ofcgsNXojCFfai?voU3tFvc)W(Jh&F4K%eGTyn zRlijj1p%rSZU`V8$|xGfDc@O2&X@H!1&~O0?tOmjgY5>Dvr$hD+rT7!{Pf%F zkOh%265@-bT`j179&LFHoJ96|Wt^;;+=&|veF;@%<2QnHHGU7~{wVn-yE8y)MUbEI zS6x!no9AJ%4~O$b17v-zwuM~2OmVO2jsIQ<0oJy*TRAv>*9)X@hg5=MA=`%p?fK6D zKrfInLNLrq3x;GqsyB$rIv@fHjf2C)C+Dgt+eas!H6YEiq)KX4AgaGfZJH%8J^f49 zsemY3G=Dh5Yp^>6wpS%5bKd!yWw3DUYNCRH%U7N#`%EXa5r;bm|Ss6be{ zG8}&eZ3R(NaLq{iAs1=SP5@4l=U!6xhbGP302*GBNYY99{qe}7j9{yb2sS!^ZD@E< zJCI_;xCcKYDk;1rB%}nzxXJM==wRJ$XatB1sbu3bMjGNEiC`;09syi%l6vvvi7Onc z?nIj8Be9(21=NJpT%?0+{a^R{Au)7=KGsRXlEl({>iE@|GurHn4qSksjFi=s{5w=e zd;ytwJ{}$_n}2o0c2S#qNcE7BQ73xqi4z|?I+)|FZETnUfaBsarwVM!E={s1PbbU! z6p*nSMjxS)*hbhMIbOx5Cnl0B)O~RZbQf8!=%ZO}_9|)gJQ7k;VFJySI(z?sg2|B) z!&9feBi!!fv^o?iCZy=7oi)0L4#H}R@0D|W?&e!kk**;x$bk`#FP(RA-XCa1;OfAK3-i6DPi5^%h>9?8!!K;ohK$UOjB zvlAyOe|^oBb)WJai8F=^rvUfU_H70SnrLb*VfcNI0P{;<2UA1SGPel;I%G3QijyKn zTM($}DL%Jj;c+0=ptq-|&6zV8vHBgbHWmOdOLAPFN&f>cpe+w#Wpc2vy!m6XbrcBP zJ^UA2z&_T~i#D)^d_C2rf4RqCW*lHv*a9dXzd}ExrK;)bO3=wd1vqUw){=r~1?k=^ zD4_jgnhLg1Y~$k-&#tzN0sF&`0_O)OLGy)4_4rhk1;>l~yE|6vMm|0}3obBo#bR%k zPEtct)9>jQ-*2&XT4!ac859tNS!U{XwkosD1?Ucb%KZ(X?@G#IcX>lGeGO%25f+Tm(x zkgUJuG1nZgM5++I3l~T;P}n*sm?WVd6++%~Mk!|>Dg8-I18>x<`~jc&pJQ~rQ0-x# zNlQz!?%1KYzB&)`^JbebX67>bBz~{2^{*dq*RkIUx(8hb(h4GIEh&mO1Qfh~fF&9( z2a%;Rr9u$QC^8KFACqy!G{mwLQd+*up6@H4n4adOrw^?7&B_Ci9vmEecV|XsX3izW zryFn&{)kqYf2*vo& zEn``knWSkCVj7Uqb4XBVzc&I_`Bc>T8NIx{hiJYMCHvdCnW;;hdU)gPk=W)F+ZCYd zqD%1)1#vH+KYY8%Z{M~OH$a$z1>o5U_Xq^4z+r0CPqH>!%CR!5y1DsrTAH%ZETkG* zszY~pNZ0h@*NF*SatwdyPaw*Ku?%~ABIyM0-o5L;P?REK`osnr4@TB*CnxVJ^_Ood zc$rzO6kaBPNca8pbO5cw>I`&opbm6SMn*;hW)8wf9Xr{F)xOGx%0zkJrRbJ11n17! zVzv(f(R3UwcK+#7++|`KqKimAQimenfA8+yufxLzu+d^-g6L&jhl%xcF8}aiW`OXb^OnvyyNMC>G+cy;V(TNH8t)9>^&YnFR(h5Z-|KtlP%(Jkvjt>rE z=B0XU9y3N1m)_>mg-<)QLWu0@QX~G0@>7&@-SEXlHhdg=Ti?)|?h=TJBRFA!U#e1gHRhvgLn8RB17f&tZeH9n;d z`Iw}nlSW1uFFBIlrmwG0uYz#8uBPTInJ5;@|3OsrgtacEk*YLXSa#bC=2;gp>;=KH zxcJ106TBf{85c(ywoYv|iX>l{bxxJSz2(7h1rf3XGj(XK^=?cg zw#SO?smb5-?_zPSG@Zc~v4FS=&f~g`=b19~bI_*{QW8xb`j1Mm3(Iz7IgARFG;~rF z9@VdNL-qYX7#bOUs_LiB4`~?A87EYoyjfMuW?^{C28n%)!@W{GVTyob;hgTI8+;V-2zaBdW0IHDDQb?(?stJRd7#J7`fQFd@-VoxGr?;v5Xf#>;;K6$9LMf+im$g+dnWW_Sc)#;x{Qt-Tgr&qH z+c-Wv{NpWV*GVx1b5qCejd906#ZhtIz&u(q_Kj5y6te&6}sKt?xU0xzJa3 zDU#a*Ifd@->ojpksGy?~0x9k6?ZH3F`uZ$_lu!bvM@B;J&h=NKlk$;H^B(nM3D9|H z%y{d{({DG+*j! zFo8gjll@ET*KVJ1LmFOg4<;Uwp283;4+gBGlacJYZEtIPBr&tL!&moz=MYZ`B+@`K zAK&*pBn{o8s5QR3ub+dA{47Ep*ZkmE6Ye!Zd)(Fh-r8|Gd9Rez+VD+X*NYcFef(H( zf$h1uXwPPzq>aXqxRr96ljR%F6p?%`zK{o5S$l+pf-helIpeTT@{|LV7Bri}r%y?t zpg4{z3qhmg+=mXV%aT(n8QysWa|L}NBjYusy8{O>H7&e*x5oK+mgCiNRQEk}v5X9i zjrTZzLR|}d1+k>OqN3*2D>-?2oIi^FnRDkv4$#Qa+}}foc`uAYHI%EWY#_XgY<4%e zy!^iD;oS0A3e&JCe)+h|mcb_Gek;?BvcT-N0z4uBTFF>SKwo?NP1W>?$@PBQ2#6GW z6pL%=kb=SkDC}p~&gU*Chhb<8;sAB1zTr;!ca#!r!zao=Y|f6|yymgt>K_n3-_8UW zaI7zkgv1J-(|;yL(3klxKQU;?O7VaA)RNG8u%+W0wc93xt*M*P3}Nn)P;3Ow95WRS zv>IB-={eMy0V@#3$+sqY;5Bfl75!J2re0%WLTEp~2qq;cN+j{@2Y}oShLm-g$T=KK z9|T9jiq?<7+6D?}p!xFVqq zQj5a`2*lZ-YGeKxbG}&Rhz0>B3DKNCcA0?$O|xU6Ps+Itt6N&8A~F@jrLeDCuh zZFXrFrJfiO_f~U)&G;w{WgvO#!@3Q%yc+U+Ds#)euxO!gA;{Tf9huvy?p=Vr z`qkA{kA6R6@!ZtZfu-3mh@I5iJjl!py>jJdJ?ZG(qOYme0Lo?VVtEF^9k381v_UE{ zeLYa25Cl_5`Hpf7n#;kVoz-4)?vrXFLF6{NRy7OafQ8+p3;_;{a%sZ#Hs5A_7Eg@-?Pn_#?I?+e?CBoJbJ@WYjC z{_Y(Ta5!d}o$l&nG1=tEXr_KkRb%#Jx*qPO)E5Xlsr@c&-sev7?vPFBW9ec)6?_p<$ za%@?&sfLCIJYNKbN-Z-ilh!~;Qb0h>)kOewJs@Bm(wP+(LiPJ(WW0f+;T+G@JfQTr zMt%}?$%EZ*k4LfRQ_eVqiY~QrQ5xSYJ7N02M;~4VAJL}F;#}UM3Br*DM2D}cssKspw^B_=w-M4p^HYioS$|ln0c$k5>9ERXa=ZdprG4#p(1*+|g@uq09Kf6HVYw3Fe%LcR~(x=I! zB;y-lRf8OU@Sh4Zpox6aA}P3NsE_$_a|b$yPMI}aqt;mh>!#AdgqQYwk z(JCGCSgkb>WUw)+tCe%|aD6TA9CM%;Plk7ULl`E;)uVdCu_Q_aw;jF&EUF}(uC6O* z8RsefamW|gT)-8<7{w}n#Lh`=3A}ze>m4cN^QU&+pWKLRhvp~*M-pFmuVuH z8y}-=OtZw_?W@{hrluP|Npv(C8WfgkR9u#7z+HH_sU z@(Hia$;rvh?FX_d9~k*tCA>3oWbtS!6U)Ph@>TplBVpI~2956aIkhI&#ua1A^rMbE z_Rj*pZ!PorUwxi`gp>ZFSeaT04~1k%SFR}fYqIC+(kvRt0eSgHQ}z-eO593Bw6DS4 zNETTavTAL#B&$y0|9fc}9Mwrrd;T2xo*ryX4DRwX@{s20IpsW4W4k-3J{`wd$x+^W z^PW^lKOgR}M@Cp(Q}ZK4T0C=O_BG;V2;~SV?rq{_iA2DyRR`{wkq6Q8Lf~ITnn98c z*$i0NsI!HbBb3_*J-)cuXgMf{DM?3-NVjFnpM>jXy#M&F`>L1q1bd&@MjG5tO8N}T zQT&j;;(srx%W>)o-d9ROLGbC?PwdrZymLvjId9>&_nY4z*zW3$7G)JS`!q$U9eVI5 z;wVq0ecPy~rq6o=#WqA{#1$3SVe7pn2Z_dGK2`w@;YyLR>4K8NThsp!3{BHD)Z6oSrPmaR0sE{pr-(URw)@sjGdbIjDpNACN7dFj@u~lWO-%hyH z5%j)p3LlSY#Jz(hipMp8QibS&;3GYRgqMm+aP0kKc*wb>~OYfjj}(U6#m;uC5q0 zP_ai@$F7D#x^WAqfxF2=21p{^-Qn68#l;t22m};NkfG`-NJ+)R%Di>!^_e0N4urMy zIxWcvmTi0iMwbK3N#F}n9+%;|7}Rp$vb8nR6x`-3>DuW#KL3*mg-2fvV%t&|NWvZ8 za)f9Oqzb)%U~Nhi^0bOuW#GYsm*!s4hzW|3pQHTa6dGtWeP$xUtw_UL_FQJGSo<}p zNu!}fN^g?MKlx_`o{tODRaoX%Q1C}m!JQ?MfwVGD@{Ot1LDNF_z_?p}e!Ezk5dkSm z@&DSou^qkgJd6=|P!h7pnN!?P8UHTI)MWzUkGU`(+F$N<-0M^4c_G$2H1=e^HU|jd ze^u#NQmn?94FwLc9}`WO8iB-Q(+TV75vj(J=pG?e)3K6)w78(sh!seIvKw;2=5e{0j@?H{cL}D4J19X3CS)|8~OFURO|lIXF%l9vS*T ziVYB2$q@1>Fv-J4#og~Ypn~U9nbiMLi2ntyI7+}lz(@&p7*>{+t#Uf)O0`NJ{>-$6 zOQBqGV_a>CXLVFGr0p2Wzxz?F_nSY3c3|E4``YP`(%%&W6$?+ky^uKYeDj5vBlMSc z+iob7NT)6AsNZ%>b-Q*t@ls4Xqrtwrysyf__uk#fnI7)O+(vus>O%=_2F>lRA`SGB z^cS}`*!ve;6N$V7BBe~A7u2k^Znj+YT=^Y95ezsu73HJ zG57EOqj&7^%r1@qE*}IIE^jxKekr6*APn5xdx;Hg24gMBUZZy~6d%BK=slr8*hP&+ z`jy`7rq8eTb7|DT=&P_GU8*IM_8C}xyeA0*8>j2Y)mViT(#$+Z`5Xj=gb)>zSx>`@ zlmuy>jxcgkRxndG^3-1Y;>zE9i&Av2j;~6I=a5w+XKa|Iav8xZY5m3c!E*=id4_Kw zyjCj6GKUV04NJAu4NA$L=&4c%8jPtTl0N+PD@Mfx-J}WJjzET+Dh-QBA1mts(L`U6 zS?&s>zNq>00t6nA18uN*&I@K!WZjpip$Ryg-pLZ~5<#y8IQ zt%RK)++L}=QjEdVv_m*@qDW9O>46hG*m0WNEl+Y5he=$JqrJ>R3ZTb5_!mc~E}hPxMAC+Z|eb%#56hBe(JMMAM_i z(KoA3td<8aN*jeF4(RO`aa6p}+gm#B;OHmjm3w-YOO2S*I3g~6(7kbPVH3kcv*$lI zd5~f*-6HJkBJm}<8z!EfyzN~u^1cMAo~GtzMis;}@R$*?lVyjA8k(Bk zQ-yD(*EVF};#7xStG;Pzo57=~7+`4`=zkLQ%Gg+qSorB128>6Sn2ZL43?WsgTlU}M zHfbMd>0etoa5dEDjS?|Uk+GaeMi@vl)(Y9lIm%(~Kx83p>Fb_L!9?qqHA*V)>4nv| z5d!(-j{Huu(!MN^=U`Ipl8NiS6e3BthqV(H;)Wfa5?CyK%I|S;35f#>hYJQ4HV~8w zNUs+TF|Upm#m`ebO^6YnF#1uTZ>3Gq;*!Z@;^G5Rw-oY;X%FjrNsiO3Mu1KqdZpY}gisrPT8BRXht22|5^@!h(WJ z?I~-o6~5si8=E$5Lc6$C@3KNg;C3UO;%mv>7$Hx%b!!!KRNNt}0~-}t+1W8I>5bG{ zh0YF6PIS6HMES6sLWsCB+{_Vz4$s}ceT4S{i|tN4pd}tK_!B|^I@rA2MZ}KE)ix6L zP;5Edv{e0D?A3_W@)HlO@WmqgggpE${58H1$SvX`kRRaLz!p?o9o!NcNkcHYP30Qe z=um5;Rlw)P?|jKc=8PCLlWk28{>V5Y_e`AiB z^rFTRL)%@dD>ceQ(<*=Yc9p6JpKyV>_cH9HbHZn+T*2ijunre=Xa(eo284eQ>aSgb zTk(=tur0Mq0uS*M7r|2oZ`Ir2q}CMhNC;FhuAvsjU#GSb2(KrtM%B+Beo8-w{KEg$ z)|ZDv+4k>`LW3|xQm95~lAVZb4N9^%2?50(yyr=hlf8XzM{Q9HgP-fS9ywyaPW14^wa@v7>n^DZi01K9n z1&K_e1t9B5nye*LQFa9`%yjT`8z5_gABwGRVIdee+)=rB=H$uoOUnewRJrMWy;N%nCZ^LkvyW>2mpmPm&dn%12^pl2DiNs(?DbjBg%m|r>t(BTVd;e z{rC$;agOVYf1jZ34tPxz^$D@DNb7Dyu(#B+4D37#9c2~qy57nlr(u0M-;+@Oz>h*t zq&uV3NFi5HT!V7B=K2d%J59PjO^3bcT>t)!P{FG1S3IQm$y#SjB%{HQij0iZ*t!+k zvnNlqSeI6=`_J#KH(pDk69h|p`?XWPkkaQee9xTW5w)i2mlm=th|sqbVT~}Ls0POk z^~^{qtSTKaB7tV4(27T(r#eiR$PR*1{G^LzUj0j^wtHaYhQ$DVrm+Rrh6&#w=ePg9 zpWyDL!Oke^p%NYGfWZk|TF`>L#AJ6^B-uPJAwf3QB$UrmbXY`v`m)8a2nRVZh*SUW zdyi5>$>Z+sN9-LG_B%K@z~AuLRSJs-AI-i1NisqUC5$0!wtM#)aq%Pe>&}fmfMgc! z9I`=+yg>LTI1Bh=+?EowIE#e@==e*RX1nM$B9h%Y3kNhpyM)cL4JJacj^O0Iy$MN; znL{=h&gasvz<&7bT{iUT9YhIaDnMajBRe^f>pM5zssm#R+CGRK2fY2!7kYi0l=3Qt z#`waz(-Z?7`~Bbzu5%w?V}Lh8M6C;BpF#L#KHHBZOp!x9}$%{$4-3y4J1oAhSuz-j`699>Gq>!j4w$ZOY# zh=XA}UTopmL%5D&AwA#^hAFq>u{S+f*=VQ5$+(Ll^XR6~#lXXa3#pI<5*cC3_uPeA*7`0%CJEzlLBc3;1er2`>lx+i|V} zGbIEoXF!Gv+k}5HM3o~xppm@s#P&u5N=&`KtxnZ{>$de1h*F(ga|UEK8d9AE|qV+j@HL+2kRtj)eAkly)CcSwY86;pDyL zDRR|F=9b;PcB7`TFJInatWEt48t(vAiGxYphn3pIF@BB7iJ@e4!vJtq%px&K05xUh zWhwB!JxcecCvm%C_N{UU=j|o9g#rRkdOVsk6yiofmjinR{{e3VUEM!H-^sC@y}1wCqRuV)Ho1~>f|FnDuxL5;@3q)~X zWVAW99+sx%J-vN>iqXa(k%nGZ?^I$ed1lGOjX*&if|KrU3yW;0?aAihHC5s(XVsp; z*u}TJXWX}Y{QH|X*0Cnix9S>?s_re`ge?q)OEq`h(&e#6%p5sIxe!jWCZ;)nM+u%d z4;5+zkeSoEM+V#>W(52aF1TJqZRZ8Z{XwX@V=8m0su|-V;4C(IP#Ufanm zi~o3rb&nstZxznJe3gO{!W zpEfX}q{cP$cxZfOa!Z8Or{lBuP@m*4hutos$IH69SqzgUedJhMR8+hHxuJ$I<4u?1QhonQRUSe9}>(i=El#<*q^!XX7fM0NY2M`H>`7~WtWhJjm%6?V7yFG=uNA!{I<+sfORjOW-=^F+CT$+5Fpt5jt&i&79=Gh2h6|SYr!MI z@Hz<27Ytfv61d6KpoLKGEBuR^t@ys8{#qY6Z3V2X zb;Q)fM9b^LGgR{OK9*IXtVu5%Z)fQ;s58Hw|7W|xvGX8bptC<(UVxXm8 zNe3AptT50Vhleynm2VjQ@0^hJprmAh1*ov!;w$y$2PY$?xKWGSY zF{?~#GYgAlnlGAFbadpSjWLFPuFfx~#X!4-j07v{v9bIP!c8L6x=uT7hm_%V#^aiC z9=zGimo}R`$e`V@NV*m@dHJ>xAA}f?O}}d~Y;9?A4X_6mnpZM_45tY)6l|Yb-(3F| zmNibDB@G;VkRqN6V25lLz5)gPGQCEMY9=h>MFWI8b_HVFzlJ zj!{%#{yeG-;`ZWDZ{L0(q*Xtd<&7acP(RGKk)j^z(IAb~0)tjmkr~FTEV$|hs-t}4 zMVU*G7t-PM71m_Y12+YCchkJK$rXoQZi6u)NIRgVZp-EfGH~(pi>Dk?Te}db1XmIG zYD3a%jpH zOLfDq*x!>`4CkeO#)yfK0WPdH;94{oHd;lRFh(tcG9tdzqjbBhEHv9!0tW&1#hSRl zyLPGyyw1ypAIscz9W=!SSO%1o4oenIR$cuOJaZeCDKmn@?s9B+UtChMS(vxr`Vnv> z+)A_nDLD!@nJz^MEKF;(1cMxefmm2?pgP12LMCNx48{?C*K>5Or2eD0N3cI0n8d&fZZF`2gBy+#CO+Qy>w2L!H6>M51J80N z_&30kPUQ~4|iV(JU^8pnjZqM!=7^ zNOG+s!`3v|X-XM{vcD69!N_6A=Cy%xCL3)`rUJ78>%pgs+sbUIa6R)BkU;ExdSBt+ z!^6(+lo z1A|LDGrr9>i~{E7$l@sPStlfP8t(TSmeGQr-P(cWfge?qUGuo_3AF@3w;Ps;dgGq0 zLowUtRsukC%wXCQdObex;3YM;_XK)*~+CIzx5=qE+x~D&D~bf`uS{p%M?K$=6SQIS!v<5E5hnC zVru5leW2T$sow(I1MkpCTYs79g%zX^b=deOnObMQ#2nigcp3_hn~uDH`<9!@2vMbA z_sy~dBb6-dPQewY-&SKd1}&V)wB8)XqMxFLuljz#9LFNU7-B=~T@7LClD z{jR2OLBfvgPtfq&p(*!Bnm-nUJg-+`6`Ts;(qTK$HtRdUEqK7Y%&|a2ba@DYC_ed{ zs><(oen@0@Z?LzmuDljQdl`4KMg%kQa;3MeqMeYa^~* zo0i?bA*_O9pm3f?iGjIZ1ws01+j z@l(;)l)DknLdPa7BxK+e#@v={e%bI1jKdjz+Ea4j(kTx`h|j$qnA>%%_u2iR)#jE0 zSNWP)AAQT=fAT3-1JW+YiBomuoue+Md3x?+Fn(x%DGBz;1YWkmZ7i`hCbi;1I<2&O zeAfM9VgLWDB^3Tu=WOL@5c*mSRXPw$+7-Hzs+#eBhV2n-Kt}6kqpxKYa6R z8OwZWaLyWbaqvDZ6N9(9qeWfgAQSs$DPPkV$PgD<&3%cLl{JXD)%1Z}#uH>M!|H3n37C9(2_f$y&7mZCU3oQPSJo?wBPqO&Koo*bc#c6&GaQnayH< z*n(}wWU}^e47`WD2PC-y&w$9Z4qZ@9qAACg5O9j{fSxC1HY*^eX7U)kOC+NU<8%;N zF^N|uX6`X2E|}j}w(#gnvETezP~jA}dwsU0K!f1frwwRApS}ZBd`2sINGBPE*cMo6 z^bLnl!`RFA;n$s4gC9R0n%k;Apo(=HM{$c@b$S%jOI5pS7>Ai;PRnC^y;!QHGZUP@|O zbqRZOO)z&SPKv=^Yw_yVOXoE6l{#aZLc7Or8%w3ciTvL#b+bc*J!)yvdQ?yQ*Ev71 zB?B@yyk{ds2i8p3a;`@;kaD80TI?@x+w^zga9NszxzD*rN$B8NmP%V6Ava-W09rdB z9Y}>Pj-trbH|7W5a)V-j@|{B^?1=yTCkLqo_+RuA!^jbhv^A zrUP$=*{nF2Z+<`P8ENz+T~fc6P^=M70l~Hfl=V{?~rji#zQ2)XRGuG1GQ%Fts=birTds zH@?qKdzE(IYjLgCjXo4&X2*xrQMj!M3y$fcW8h31;x|$=_#vRN&vB1m>E835qCFK> z_&(+r4Wk`0mE+Atw0!7FbPZNocE+&xagtKvq)yj3ar20rwf4G_ zt0BG-2xjz`D5e9~VZ#FQD~dnb$rt&1t5TICJ%}(|S++yUvi;&oSJwy%!0E9mSEyI0 zAzi0wHxoqO)+?U40BC4kSVzq|BLNrr060tmrE)1v)eCaWWek4)-0W$zul3?b%XY~a z>Ty!Aw84u)T1-hA4INwkz$8p%nqJ_q5_<_TF;v$^mkYAEN^I@;>|frlq2oO6v2k*W zyhc^ROW^QXH!ukTVP4z2ckiBHsz`qczC2_~n;*0nr_INeq(z2#vs3Sc((|IX->7N% zq1o(a#kk=1R?+3CE2gb2l{<>l0%A+jx(zDCMX)D>U2r(vJxUYr$JbS?@;BqVvakJO zdQP)qcqqQvkBv8tCTS;Dh`QPT5Skk(l;R`>ul{cr4K~wit=k_6JFF8Ywv;a*sRDb! zmI5LQrx%DS(B%5qT}8R`ZGHBJVP-Q{Y42WDSQkNOmeJw|$GyQ248KA{x!@#*ko^O$ zPiQv)KXu@%=>lzM#B8+gUG4%uNBL9G<}=mSCYv8aHizux`*f=X zpG34XDEDS&uV8C2Qfi&+1B?tO1!n}B8)I|xl*Gi*u`v*6DC*{mMmu+|qyzA3${+~M z*|?rt=eJ`528QOuYH@?kBrcN8eLxJT7;n`H|4Xuhgd2_<-vEL+jObNNs$&7CW?*h9d`Vh_eQcSy25Cg_7y1FUJ8m=Yw z%I^1t_A$exc`RG$N%Y@~=*Q5V;{Lvhk?LOqhAqNyt=ChI0ABn?F};kv%(mPE)#agDO~&&WDz-jJTU0TA2VK@X@JYC+pmpS&OPMJtQV*2&d#u>>w-O zF$Ij4mv=R?(9hVEq@VP=MGZ(;|7tXye)OnJdzC)|L(uc@;wQiS403srAG&|dD;RuL z@3*o-ET9-JPI6ly2%w1mEHB0KQVbv1su=c7T_}e9N%C$zlQ>o%c#|{$(h1SgD#e#V z7QigKjbPTf?K_W|c3I5oqOP%-j$&K65Xxo4belXLD$))_Ktr2EiBIt1m<)^5oij$E ztgfD5ngw<*J;GFm5a$NWs{A^|Emm(|D8Dssa_oZF_$?zITLC^~rHLwcbx_&L9qvN` z1u5P`#3y-qvrOw2r}&SbKAFk1!X6wv*C8LRPyzJ_1d;3K6l#07ZFpU=;~9H(gEyf^ zo6g_^|2y^0o#X0>;L@VWe1g6>btrr^tY(+oiK-L%0sS}k05y7$nqg)IXWoDCwthUS zQse`_FC!a%jd=+VTU@n$hABTACmf}kQZ|+$8qj3r8kq74ds;UAXcWttFjMXdk!K%j z&QIQD$mEc6>@OD;sXG^2qmuZ_yDaJH*rnHED0{WVYD!RFN>|Sh@b@PlI&0^qnLl8b zO%xN(_(HqLJlY%~-^@m_$q_;a<^OspB5Z_83nua0I=a-tpU+2Au~#FgwbO@+y0)d1 zjg1H+(olf*TsP02a$nIFW}vR%B2R0Ht4Tn&-O!MsvF6IA+5mEu@$XN0SX6}J`khkA z_g*iFJ|yA_8itVh7u+hp1Mu zHIw0oulPenz;S8$O6Knj-!Ff>hLwXYB%PuYF21)B?h`0J9~|@~`?3?y25ki~0SSR! zSQiN3TU=*cVRH07+uH9w~ux-zcICCE;c`o`pn{=)L|1KF*+rEcQ+4Y%sx zFpQjzS!hT^%zlS@=H-)3{Wo#CWIaEy*kuWM6}EF;n@QiJf0A?|LqN^lv3$k%&Vj<` zWkXTt%=k*XlhGRA?{MyvuD*D%otTD&P`)1lQxZ&G%<;RXcA!nN&P_^4_^7dy>K|LP zvOmRuJML-jp~u{x_6al$@if@}ewj!?-3D;P^V&dydD!Gr-m~_47Hp&YNTHwY1okAI z6VTy$E%w}>UnbaPw{o-^N&h8NT^uN(5Pb=w2oDF_2D)%0i|5+r@X&`MYC-;&D(PI1 zlt8H2Cu&gbof(f(e1i|{7*^1snx0RF#Xx&=^Rr{d)#(jPR(|LvUo$?daq0d;YxbHF z=j;h+rDW>+^-UXL#~i-l2c7<4PA8{^o?dEdY9BkFoQ7P#AOi(9;g;tZ@U0eOBynxO31Au*IqOt?!<7sO zl6ha?KN3B}j17qczzvY&Wpk&h8ay!ANv=HbtOL*myuM(6Y@j3;DzL0;e;NLFtIkl4ngjz^=GAZr< zWHcGVr9f=cn7OpaNV;4K>;q`CJiLZk+I=dsN~oSG&zq_1+F;E-%+KlvONJfUNAK`T_8A+}-nWpwDt2 zoT)gO8Q-NnSzu7VA;@PW|7xq^dxzQc}R??R~X-?cG9COlgQw?Dv-h~ayMGuzxohW>7R^$OjEaY0AJz$7MQlatqwLj6_Q3Nd zaQEQH$$L~UG{k7g_yLc@m!)eAf-7~)-!4OtvV~_HC-wQz){}7!nU*y4z_crGT-6}| zMsbKr55>e=S2ZwmSr*&9KS!8kVspXZ-}U%$^mkaPX)0!I08wpiT_4z@SAm=yS|yww z2pz$3-m_R<^#;#fyL&g@R3>Xe{+1d@e6DH#ZP!PI@xB=ASTL`IHW$?wYGdF|EJmw$aBuuYKx&Kwry+fi<2Bz*> zEg~Xy^Vhr#CJ@Ko0n_R6_y^wyU?aG|I$ssKx|4%{AYj>6b%|EPn_Y-_})bE%AMnatn^#Su8 zA&+SI!7oU%S@+b@f?Ni6&Cg~GFbU>usy^8&udR`a+d)31l{~Hes&@)dsgWn&^!!`c z4*+B&9?_$tGmg=U#~&cB26tlARdIgF!dYM6cGcx;Z@rJ;(AfnNKu5oq(l31ql*%hs zRP{Q#7Ij_Q`3V~yqjErmz>sc6N3*akG3g8j30W{V)xT$>x`v08Smo@jw>_~=3@C?TqH%UIT2@)34Rz*!qa6ADQyE|zCWS1{0R~hj!%0>$5*(& z0!~K}tN<7|KmYCM43-;*9VZFk7ST*RU5r7Ac-hc<6hP0ZeDp|t^XB6~yVMiMGa_}hP^|rVz<`8(HPxl7s`|-C3$s)s zT=%cu4^yRKJms56!FfP6M6d5e^TLotKY~I@XT2AIGRR%)-yZKQIu=>epC(w=J`4(HXZ`0Wh@B)#33?^`TEOi&(BS_X=fOXfLxm5lj4WY9o0OZ0 z#%!BS$XLQ;m;az>=z}n_3tc^%hjd~7dub(}#)CQIPWC?}IkU;;zxhF=ZL@l*0&C6& UUy1Dlse`oJ$lUNg-TC7G06m{b(EtDd diff --git a/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv b/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv index 6e4a5a479..de57d57fc 100644 --- a/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv +++ b/docs/modules/ROOT/pages/performance/sql_and_scan_query_perf_comp.csv @@ -1,5 +1,7 @@ -OPERATION TYPE MIN MAX AVG MEDIAN MODE P99 -SQL_QUERY_EXEC 1.0000 16.0000 1.8739 1.0000 1.0000 1.0000 -SCAN_QUERY_EXEC 530182.0000 785735.0000 561550.7678 552705.0000 543704.0000 531308.3565 +OPERATION TYPE MIN MAX AVG MEDIAN MODE P99 +SQL_QUERY_EXEC 1.0000 16.0000 1.8739 1.0000 1.0000 1.0000 +SCAN_QUERY_EXEC 530182.0000 785735.0000 561550.7678 552705.0000 543704.0000 531308.3565 SQL_CURSOR_FETCH 0.0000 1.0000 0.6717 1.0000 1.0000 0.0000 SCAN_CURSOR_FETCH 0.0000 8.0000 0.1411 0.0000 0.0000 0.0000 +SQL_INSERT 0.0000 929676278.00 929.676278 0.0 0.0 0.0 +SCAN_INSERT 0.0000 812451365.00 812.451365 0.0 0.0 0.0 diff --git a/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc index b387046a4..eabce1bcb 100644 --- a/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc +++ b/docs/modules/ROOT/pages/performance/sql_and_scan_query_pref_comp.adoc @@ -19,7 +19,7 @@ Number of entries in both SQL and Scan are 1M, 1000 entries are queried on. The first query always takes way too much time compared to the subsequent ones and hence ignored when computing the statistics. -All times in micro seconds. +All times are in micro seconds. Time axis (y) in the histogram is in logscale. SQL_QUERY_EXEC time represents time required to execute the SQLFieldsQuery (cache.query() API). @@ -33,13 +33,17 @@ instantiation, cache.withKeepBinary().query(...) SCAN_CURSOR_FETCH time represents time required to extract the result from result set (cursor, cursor.getAll() API). +For Insert operation, MAX represents total time to insert 1M entires into the cache and AVG represents the average time to insert one single entry. All other statistcs for Insert are not relevant and are shown as zero. + == Results |=== |OPERATION TYPE| MIN| MAX| AVG| MEDIAN| MODE| P99 |SQL_QUERY_EXEC| 1.0000| 16.0000| 1.8739| 1.0000| 1.0000| 1.0000 -|SQL_CURSOR_FETCH| 0.0000| 1.0000| 0.6717| 1.0000| 1.0000| 0.0000 |SCAN_QUERY_EXEC| 530182.0000| 785735.0000| 561550.7678| 552705.0000| 543704.0000| 531308.3565 +|SQL_CURSOR_FETCH| 0.0000| 1.0000| 0.6717| 1.0000| 1.0000| 0.0000 |SCAN_CURSOR_FETCH| 0.0000 | 8.0000 | 0.1411| 0.0000| 0.0000| 0.0000 +|SQL_INSERT| 0.0000 | 929676278.00| 929.6762| 0.0| 0.0| 0.0 +|SCAN_INSERT| 0.0000| 812451365.00| 812.4514| 0.0| 0.0| 0.0 |=== == Plot of the results diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index c5ac8a2fe..80e910fb2 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -64,9 +64,7 @@ private static class SqlField { public String type; public boolean isIndexed; } - private Map sqlFields = null; // needed for index creation and querying - private List sqlColumns = null; // needed for constructing indexes in specified order - // so that a prefix query can use indexed search. + private LinkedHashMap sqlFields = null; // needed for index creation and querying private boolean checkedForSqlFields = false; public IgniteClientDbCache(IgniteClient igniteClient, Class v, String name) { @@ -350,8 +348,8 @@ private String buildSqlFieldsQuery(Map filterParams) { StringBuilder sb = new StringBuilder("select _key, _val from " + SQL_SCHEMA_NAME + "." + cache.getConfiguration().getQueryEntities()[0].getTableName() + " where "); boolean needAnd = false; - for (String c : sqlColumns) { - if (!sqlFields.get(c).isIndexed || !filterParams.containsKey(c)) + for (String c : filterParams.keySet()) { + if (!sqlFields.get(c).isIndexed) continue; SqlField f = sqlFields.get(c); if (needAnd) @@ -395,11 +393,12 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str LinkedHashMap qryFields = new LinkedHashMap<>(); ArrayList idxFields = new ArrayList<>(); try { - for (String c : sqlColumns) { - SqlField f = sqlFields.get(c); - qryFields.put(c, f.type); + for (Map.Entry e : sqlFields.entrySet()) { + String cname = e.getKey(); + SqlField f = e.getValue(); + qryFields.put(cname, f.type); if (f.isIndexed) { - idxFields.add(new QueryIndex(c)); + idxFields.add(new QueryIndex(cname)); } } @@ -435,7 +434,6 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str private void extractSqlFields(String className) { Map localFields = new HashMap<>(); - List localColumns = new ArrayList<>(); logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + className); Field[] fields = null; try { @@ -457,13 +455,10 @@ private void extractSqlFields(String className) { sqlField.type = f.getType().getTypeName(); localFields.put(f.getName(), sqlField); } - - localColumns.add(f.getName()); } } sqlFields = localFields; - sqlColumns = localColumns; logger.log(Level.INFO, "QuerySqlField Found " + sqlFields.size() + " sqlFields"); } catch (Exception e) { logger.log(Level.INFO, "Failed to get Class info for class " + className + ", or declared fields"); diff --git a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java index 980bcd9b9..c72e698b8 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java +++ b/lib/src/main/java/com/futurewei/alcor/common/entity/CustomerResource.java @@ -24,7 +24,6 @@ free of charge, to any person obtaining a copy of this software and associated d public class CustomerResource extends Resource { @JsonProperty("project_id") - // @QuerySqlField(index = true) private String projectId; @JsonProperty("tenant_id") @@ -112,4 +111,4 @@ public String toString() { ", description='" + description + '\'' + '}'; } -} +} \ No newline at end of file diff --git a/services/scanquery/pom.xml b/services/scanquery_test_nodemanager/pom.xml similarity index 100% rename from services/scanquery/pom.xml rename to services/scanquery_test_nodemanager/pom.xml diff --git a/services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java b/services/scanquery_test_nodemanager/src/main/java/com/futurewei/alcor/scanquery/scanquery.java similarity index 100% rename from services/scanquery/src/main/java/com/futurewei/alcor/scanquery/scanquery.java rename to services/scanquery_test_nodemanager/src/main/java/com/futurewei/alcor/scanquery/scanquery.java diff --git a/services/sqlquery/pom.xml b/services/sqlquery_test_nodemanager/pom.xml similarity index 100% rename from services/sqlquery/pom.xml rename to services/sqlquery_test_nodemanager/pom.xml diff --git a/services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java b/services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java similarity index 100% rename from services/sqlquery/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java rename to services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java diff --git a/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java b/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java index 2157731fb..fec60adcb 100644 --- a/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java +++ b/web/src/main/java/com/futurewei/alcor/web/entity/port/PortEntity.java @@ -20,7 +20,6 @@ free of charge, to any person obtaining a copy of this software and associated d import com.futurewei.alcor.common.entity.CustomerResource; import lombok.Data; import lombok.NoArgsConstructor; -import org.apache.ignite.cache.query.annotations.QuerySqlField; import java.util.List; @@ -28,7 +27,6 @@ free of charge, to any person obtaining a copy of this software and associated d @JsonInclude(JsonInclude.Include.NON_ABSENT) public class PortEntity extends CustomerResource { @JsonProperty("network_id") - @QuerySqlField(index = true) private String vpcId; @JsonProperty("admin_state_up") From eb19113bdef0cf9313741f6372dcdb998f8562b8 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 1 Dec 2021 16:08:04 -0800 Subject: [PATCH 26/27] Fix compile error and IntelliJ issues --- .../futurewei/alcor/common/db/ignite/IgniteClientDbCache.java | 2 +- .../src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java index 80e910fb2..2519a591b 100644 --- a/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java +++ b/lib/src/main/java/com/futurewei/alcor/common/db/ignite/IgniteClientDbCache.java @@ -433,7 +433,7 @@ private ClientCache getOrCreateIndexedCache(IgniteClient igniteClient, Str private void extractSqlFields(String className) { - Map localFields = new HashMap<>(); + LinkedHashMap localFields = new LinkedHashMap<>(); logger.log(Level.INFO, "Checking for QuerySqlField annotations: " + className); Field[] fields = null; try { diff --git a/services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java b/services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java index 68c9a5377..cf630e17f 100644 --- a/services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java +++ b/services/sqlquery_test_nodemanager/src/main/java/com/futurewei/alcor/sqlquery/sqlquery.java @@ -291,4 +291,4 @@ static String getNextMAC() return newMac; } -} +} \ No newline at end of file From 3cf7110cbeb759076f02b8d77f4ad5d13b815c09 Mon Sep 17 00:00:00 2001 From: Prasad Kommoju Date: Wed, 1 Dec 2021 16:45:33 -0800 Subject: [PATCH 27/27] Set logging level to default --- services/node_manager/src/main/resources/application.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/services/node_manager/src/main/resources/application.properties b/services/node_manager/src/main/resources/application.properties index 17b8084ad..90b89a21d 100644 --- a/services/node_manager/src/main/resources/application.properties +++ b/services/node_manager/src/main/resources/application.properties @@ -35,7 +35,7 @@ ignite.thin.client.enable=true #####Logging configuration##### #logging.file.path=./ #logging.file.name=node-manager.log -logging.level.root=DEBUG +#logging.level.root=INFO ## Misc: MULTIPART (MultipartProperties) # Enable multipart uploads