Skip to content

Commit

Permalink
Merge pull request #161 from zzq996/master
Browse files Browse the repository at this point in the history
Version 3.24.8
  • Loading branch information
zzq996 authored Aug 31, 2024
2 parents 5af1b31 + ab7c0f8 commit 9ff77dd
Show file tree
Hide file tree
Showing 56 changed files with 2,402 additions and 153 deletions.
18 changes: 18 additions & 0 deletions README-Android.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
Version 3.24.8
Resolved issues:
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
2. UploadFile can be cancelled and aborted now.
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
5. Optimised the logic of set progress listener when using uploadFile.
6. Optimised some log info format.
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.

Third-party dependence:
1. Replace okio 3.8.0 with okio 3.6.0
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
-----------------------------------------------------------------------------------
Version 3.24.3
Resolved issues:
1. Optimized log info of some exception stack
Expand Down
18 changes: 18 additions & 0 deletions README-Java.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
Version 3.24.8
Resolved issues:
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
2. UploadFile can be cancelled and aborted now.
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
5. Optimised the logic of set progress listener when using uploadFile.
6. Optimised some log info format.
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.

Third-party dependence:
1. Replace okio 3.8.0 with okio 3.6.0
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
-----------------------------------------------------------------------------------
Version 3.24.3
Resolved issues:
1. Optimized log info of some exception stack
Expand Down
18 changes: 18 additions & 0 deletions README.MD
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
Version 3.24.8
Resolved issues:
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart now supports crc64 checksum.
2. UploadFile can be cancelled and aborted now.
3. Allow you set okhttp's EventListenerFactory to profile each stage of a http request,not set by default.
4. Fixed the issue that client encryption is available only in obs protocal and add some check logic when encryption algrithm is null.
5. Optimised the logic of set progress listener when using uploadFile.
6. Optimised some log info format.
7. Added some logic to compatible with Android 7.0 when using DateTimeFormatter.

Third-party dependence:
1. Replace okio 3.8.0 with okio 3.6.0
2. Replace log4j-core 2.20.0 with log4j-core 2.18.0
3. Replace jackson-core 2.15.4 with jackson-core 2.15.2
4. Replace jackson-databind 2.15.4 with jackson-databind 2.15.2
5. Replace jackson-annotations 2.15.4 with jackson-annotations 2.15.2
6. Replace log4j-api 2.20.0 with log4j-api 2.18.0
-----------------------------------------------------------------------------------
Version 3.24.3
Resolved issues:
1. Optimized log info of some exception stack
Expand Down
18 changes: 18 additions & 0 deletions README_CN.MD
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
Version 3.24.8
Resolved issues:
1. PutObject、Getobject、GetObjectMetadata、UploadPart、AppendObject、CopyObject、CopyPart、CompeleMultiUploadPart支持crc64校验
2. 断点续传上传支持暂停、取消
3. 支持设置okhttp的EventListenerFactory,用于统计http请求各阶段耗时,默认关闭
4. 修复客户端加密只能在obs协议下使用的问题,增加加密算法为null时的判断
5. 优化断点续传上传时的进度条设置逻辑
6. 优化部分日志打印格式
7. 使用DateTimeFormatter时兼容Android 7.0

Third-party dependence:
1. 使用 okio 3.8.0 替代 okio 3.6.0
2. 使用 log4j-core 2.20.0 替代 log4j-core 2.18.0
3. 使用 jackson-core 2.15.4 替代 jackson-core 2.15.2
4. 使用 jackson-databind 2.15.4 替代 jackson-databind 2.15.2
5. 使用 jackson-annotations 2.15.4 替代 jackson-annotations 2.15.2
6. 使用 log4j-api 2.20.0 替代 log4j-api 2.18.0
-----------------------------------------------------------------------------------
Version 3.24.3
Resolved issues:
1. 优化某些堆栈的日志打印
Expand Down
4 changes: 2 additions & 2 deletions app/src/main/java/com/obs/log/LoggerBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ static class GetLoggerHolder {
try {
loggerClass = Class.forName("java.util.logging.Logger");
getLoggerClass = GetLoggerHolder.loggerClass.getMethod("getLogger", String.class);
} catch (NoSuchMethodException | SecurityException | ClassNotFoundException |
NoClassDefFoundError exx) {
} catch (NoSuchMethodException | SecurityException | ClassNotFoundException
| NoClassDefFoundError exx) {
ILOG.warning(exx.getMessage());
}
}
Expand Down
3 changes: 2 additions & 1 deletion app/src/main/java/com/obs/services/AbstractClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ protected void init(String accessKey, String secretKey, String securityToken, Ob
if (this.isAuthTypeNegotiation()) {
this.getProviderCredentials().setIsAuthTypeNegotiation(true);
}
this.initHttpClient(config.getHttpDispatcher(), config.getCustomizedDnsImpl(), config.getHostnameVerifier());
this.initHttpClient(config.getHttpDispatcher(), config.getCustomizedDnsImpl(), config.getHostnameVerifier(),
config.getEventListenerFactory());
OBSXMLBuilder.setXmlDocumentBuilderFactoryClass(config.getXmlDocumentBuilderFactoryClass());
reqBean.setRespTime(new Date());
reqBean.setResultCode(Constants.RESULTCODE_SUCCESS);
Expand Down
12 changes: 12 additions & 0 deletions app/src/main/java/com/obs/services/IObsClientAsync.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package com.obs.services;

import com.obs.services.internal.task.UploadFileTask;
import com.obs.services.model.CompleteMultipartUploadResult;
import com.obs.services.model.TaskCallback;
import com.obs.services.model.UploadFileRequest;

public interface IObsClientAsync {
UploadFileTask uploadFileAsync(
UploadFileRequest uploadFileRequest,
TaskCallback<CompleteMultipartUploadResult, UploadFileRequest> completeCallback);
}
159 changes: 159 additions & 0 deletions app/src/main/java/com/obs/services/ObsClientAsync.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
package com.obs.services;

import com.obs.log.ILogger;
import com.obs.log.LoggerBuilder;
import com.obs.services.internal.task.UploadFileTask;
import com.obs.services.model.CompleteMultipartUploadResult;
import com.obs.services.model.TaskCallback;
import com.obs.services.model.UploadFileRequest;

import java.io.IOException;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class ObsClientAsync extends ObsClient implements IObsClientAsync {
/**
* Constructor
*
* @param endPoint OBS endpoint
*/
public ObsClientAsync(String endPoint) {
super(endPoint);
}

/**
* Constructor
*
* @param config Configuration parameters of ObsClient
*/
public ObsClientAsync(ObsConfiguration config) {
super(config);
}

/**
* Constructor
*
* @param accessKey AK in the access key
* @param secretKey SK in the access key
* @param endPoint OBS endpoint
*/
public ObsClientAsync(String accessKey, String secretKey, String endPoint) {
super(accessKey, secretKey, endPoint);
}

/**
* Constructor
*
* @param accessKey AK in the access key
* @param secretKey SK in the access key
* @param config Configuration parameters of ObsClient
*/
public ObsClientAsync(String accessKey, String secretKey, ObsConfiguration config) {
super(accessKey, secretKey, config);
}

/**
* Constructor
*
* @param accessKey AK in the temporary access key
* @param secretKey SK in the temporary access key
* @param securityToken Security token
* @param endPoint OBS endpoint
*/
public ObsClientAsync(String accessKey, String secretKey, String securityToken, String endPoint) {
super(accessKey, secretKey, securityToken, endPoint);
}

/**
* Constructor
*
* @param accessKey AK in the temporary access key
* @param secretKey SK in the temporary access key
* @param securityToken Security token
* @param config Configuration parameters of ObsClient
*/
public ObsClientAsync(String accessKey, String secretKey, String securityToken, ObsConfiguration config) {
super(accessKey, secretKey, securityToken, config);
}

public ObsClientAsync(IObsCredentialsProvider provider, String endPoint) {
super(provider, endPoint);
}

public ObsClientAsync(IObsCredentialsProvider provider, ObsConfiguration config) {
super(provider, config);
}

private static final ILogger log = LoggerBuilder.getLogger(ObsClientAsync.class);
private ExecutorService asyncClientExecutorService;
private static final int DEFAULT_CLIENT_EXECUTOR_SERVICE_SIZE = 128;

private int queryInterval = 1000;

@Override
public void close() throws IOException {
log.warn("ObsClientAsync closing");
try {
// finishing all task
getExecutorService().shutdown();
log.warn("ObsClientAsync closed");
} catch (Exception e) {
log.warn("ObsClientAsync close failed, detail:", e);
}
super.close();
}

private static final String ASYNC_CLIENT_EXECUTOR_SERVICE_THREAD_NAME = "async-client-thread";
protected ExecutorService getExecutorService() {
if (asyncClientExecutorService == null) {
asyncClientExecutorService = Executors.newFixedThreadPool(DEFAULT_CLIENT_EXECUTOR_SERVICE_SIZE,
r -> new Thread(r, ASYNC_CLIENT_EXECUTOR_SERVICE_THREAD_NAME));
}
return asyncClientExecutorService;
}

public void setExecutorService(ExecutorService service) {
if (asyncClientExecutorService != null) { // wait for all task finish
asyncClientExecutorService.shutdown();
while (!asyncClientExecutorService.isTerminated()) {
try {
Thread.sleep(queryInterval);
} catch (InterruptedException e) {
log.warn("ObsClientAsync setExecutorService failed, detail:", e);
}
}
}
asyncClientExecutorService = service;
}

public int getQueryInterval() {
return queryInterval;
}

public void setQueryInterval(int queryInterval) {
this.queryInterval = queryInterval;
}

/**
* @param uploadFileRequest
* @param completeCallback
* @return
*/
@Override
public UploadFileTask uploadFileAsync(
UploadFileRequest uploadFileRequest,
TaskCallback<CompleteMultipartUploadResult, UploadFileRequest> completeCallback) {
log.debug("start uploadFileAsync");
if (uploadFileRequest.getCancelHandler() != null) {
uploadFileRequest.getCancelHandler().resetCancelStatus();
}
UploadFileTask uploadFileTask =
new UploadFileTask(this, uploadFileRequest.getBucketName(), uploadFileRequest, completeCallback);
Future<?> future = getExecutorService().submit((Callable<?>) uploadFileTask);

uploadFileTask.setResultFuture(future);
return uploadFileTask;
}
}
13 changes: 13 additions & 0 deletions app/src/main/java/com/obs/services/ObsConfiguration.java
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

import okhttp3.Dispatcher;
import okhttp3.Dns;
import okhttp3.EventListener;

import java.security.SecureRandom;

Expand Down Expand Up @@ -104,6 +105,7 @@ public class ObsConfiguration implements Cloneable {
private HostnameVerifier hostnameVerifier;

private String xmlDocumentBuilderFactoryClass;
private EventListener.Factory eventListenerFactory;

/**
* Constructor
Expand Down Expand Up @@ -939,4 +941,15 @@ public SecureRandom getSecureRandom() {
public void setSecureRandom(SecureRandom secureRandom) {
this.secureRandom = secureRandom;
}

public EventListener.Factory getEventListenerFactory()
{
return eventListenerFactory;
}

public void setEventListenerFactory(EventListener.Factory eventListenerFactory)
{
this.eventListenerFactory = eventListenerFactory;
}

}
14 changes: 9 additions & 5 deletions app/src/main/java/com/obs/services/crypto/CryptoObsClient.java
Original file line number Diff line number Diff line change
Expand Up @@ -339,16 +339,20 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
// 该接口是下载对象,需要将流返回给客户(调用方),我们不能关闭这个流

if (ctrCipherGenerator != null) {
String headerMetaPrefix =
this.getProviderCredentials() != null &&
this.getProviderCredentials().getLocalAuthType(request.getBucketName()) != AuthTypeEnum.OBS
? Constants.V2_HEADER_META_PREFIX : Constants.OBS_HEADER_META_PREFIX;
String encryptedAlgorithm =
(String)
objMetadata
.getOriginalHeaders()
.get(Constants.OBS_HEADER_META_PREFIX + ENCRYPTED_ALGORITHM_META_NAME);
.get(headerMetaPrefix + ENCRYPTED_ALGORITHM_META_NAME);
String encryptedStart =
(String)
objMetadata
.getOriginalHeaders()
.get(Constants.OBS_HEADER_META_PREFIX + ENCRYPTED_START_META_NAME);
.get(headerMetaPrefix + ENCRYPTED_START_META_NAME);

if (isValidEncryptedAlgorithm(encryptedAlgorithm)) {
byte[] cryptoKeyBytes = ctrCipherGenerator.getCryptoKeyBytes();
Expand All @@ -365,7 +369,7 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
objMetadata
.getOriginalHeaders()
.get(
Constants.OBS_HEADER_META_PREFIX
headerMetaPrefix
+ ENCRYPTED_AES_KEY_META_NAME);
// 解密rsa加密后的主密钥
cryptoKeyBytes =
Expand Down Expand Up @@ -425,8 +429,8 @@ protected ObsObject getObjectImpl(GetObjectRequest request) throws ServiceExcept
}

public boolean isValidEncryptedAlgorithm(String encryptedAlgorithm) {
return encryptedAlgorithm.equals(CtrRSACipherGenerator.ENCRYPTED_ALGORITHM)
|| encryptedAlgorithm.equals(CTRCipherGenerator.ENCRYPTED_ALGORITHM);
return encryptedAlgorithm != null && (encryptedAlgorithm.equals(CtrRSACipherGenerator.ENCRYPTED_ALGORITHM)
|| encryptedAlgorithm.equals(CTRCipherGenerator.ENCRYPTED_ALGORITHM));
}

protected byte[] getOrGenerateCryptoIvBytes() {
Expand Down
Loading

0 comments on commit 9ff77dd

Please sign in to comment.