Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 180 additions & 0 deletions docs/design/kangcheolung-#44-search-embedding-query-logging.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# #44 검색 블록 — 질문 임베딩 + 검색 요청 로깅 (F-SEARCH-01/02/03)

closes #44

## 배경

벡터 검색의 첫 단계는 사용자의 검색어를 임베딩 모델과 동일한 차원의 벡터로 변환하는 것이다.
이번 이슈에서는 Python 사이드카 서버(`/embed`)를 호출해 질문을 1024차원 벡터로 변환하는 서비스(F-SEARCH-01/02)와,
검색 요청 자체를 `search_queries` 테이블에 `PROCESSING → SUCCESS/FAILED` 흐름으로 로그 남기는 서비스(F-SEARCH-03)를 구현한다.

이 이슈는 서비스 레이어 "부품" 구현에 집중하며, `POST /search` API 조립은 Issue 5에서 진행한다.

---

## 작업 내용

### 1. 기존 파일 수정

#### `ResultStatus.java`

`PROCESSING` 값 추가. 검색 요청 저장 시 초기 상태로 사용한다.

```java
public enum ResultStatus {
PROCESSING, // 추가
SUCCESS,
FAILED
}
```

DB는 `VARCHAR(20)` CHECK 없음 → Flyway 마이그레이션 불필요.

#### `ErrorCode.java`

검색 블록 에러 코드 2종 추가.

| 코드 | HTTP | 설명 |
|------|------|------|
| `EMBEDDING_SERVER_UNAVAILABLE` | 503 | Python 서버 타임아웃/연결 실패 |
| `EMBEDDING_DIMENSION_MISMATCH` | 500 | 응답 차원 ≠ 모델 설정 차원 |

#### `application.yml`

```yaml
embedding:
server:
base-url: ${EMBEDDING_SERVER_URL:http://localhost:8000}
```

#### `SearchQuery.java`

dirty checking 기반 상태 갱신 메서드 추가.

```java
public void updateToSuccess(int latencyMs) {
this.status = ResultStatus.SUCCESS;
this.latencyMs = latencyMs;
}

public void updateToFailed(String errorMessage) {
this.status = ResultStatus.FAILED;
this.errorMessage = errorMessage;
}
```

명시적 `save()` 없이 트랜잭션 종료 시점에 dirty checking으로 자동 반영된다.

---

### 2. EmbeddingServerConfig — RestClient Bean 등록

`global/config/EmbeddingServerConfig.java`

```java
@Bean("embeddingRestClient")
public RestClient embeddingRestClient() { ... }
```

- JDK HttpClient 기반, connectTimeout = readTimeout = 5s
- `@Qualifier("embeddingRestClient")`로 주입해 다른 RestClient Bean과 충돌 방지

---

### 3. Python 서버 통신 DTO

| 클래스 | 역할 |
|--------|------|
| `EmbedRequest(String text)` | `POST /embed` 요청 바디 |
| `EmbedServerResponse(float[] vector, int dimension)` | 응답 파싱 |
| `EmbedResult(EmbeddingModel model, float[] vector)` | 서비스 간 전달용 내부 record |

`EmbedResult`는 Issue 5에서 `SearchQueryCommandService.createProcessing()`에 그대로 전달된다.

---

### 4. QueryEmbeddingService (F-SEARCH-01/02)

`domain/embedding/service/query/QueryEmbeddingService.java`

**`embed(String text)` 처리 흐름:**

```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

코드 펜스에 언어 식별자를 추가하세요.

markdownlint-cli2의 MD040 경고가 발생한 네 블록에 text 또는 적절한 언어 태그를 지정해야 합니다.

수정 예시
-```
+```text

Also applies to: 127-127, 136-136, 144-144

🧰 Tools
🪛 markdownlint-cli2 (0.23.0)

[warning] 102-102: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/design/kangcheolung-`#44-search-embedding-query-logging.md at line 102,
문서의 언어 식별자가 없는 네 개의 코드 펜스를 찾아 각각 ```text 또는 내용에 맞는 언어 태그를 추가하세요. 기존 코드 블록 내용과 문서
구조는 유지하고 markdownlint MD040 경고가 발생하지 않도록 수정하세요.

Source: Linters/SAST tools

1. EmbeddingModelQueryService.getActiveModel()
└─ 0건 → EMBEDDING_MODEL_NOT_CONFIGURED (500) ← 기존 구현 재사용
└─ 2건 이상 → MULTIPLE_ACTIVE_EMBEDDING_MODELS (500) ← 기존 구현 재사용

2. POST /embed { "text": text } 호출
└─ RestClientException (타임아웃/연결 실패) → EMBEDDING_SERVER_UNAVAILABLE (503)

3. 응답 차원 검증
└─ response.dimension() != model.getDimension() → EMBEDDING_DIMENSION_MISMATCH (500)

4. EmbedResult(model, vector) 반환
```

- `@Transactional` 없음 — DB 접근 없이 외부 HTTP 호출만 수행
- `RestClientException`으로 타임아웃/연결 거부를 통합 처리 → Spring 본체 격리 보장

---

### 5. SearchQueryCommandService (F-SEARCH-03)

`domain/search/service/command/SearchQueryCommandService.java`

#### `createProcessing()`

```
- user, collection(nullable), queryText, model, vector, topK 받아서
- searchType = VECTOR 고정 (MVP)
- status = PROCESSING 으로 save
- 저장된 SearchQuery 반환 (query_id가 이후 search_results / RAG 블록의 부모 키)
```

#### `markSuccess(SearchQuery, int latencyMs)`

```
- status → SUCCESS
- latencyMs 갱신
- dirty checking 자동 반영 (명시적 save 없음)
```

#### `markFailed(SearchQuery, String errorMessage)`

```
- status → FAILED
- errorMessage 갱신
- dirty checking 자동 반영
- 임베딩 서버 장애, 권한 오류 등 검색 전 단계 실패도 여기서 처리
```

---

## 에러 케이스 정리

| 상황 | 예외 | HTTP |
|------|------|------|
| active 임베딩 모델 없음 | `EMBEDDING_MODEL_NOT_CONFIGURED` | 500 |
| active 임베딩 모델 2개 이상 | `MULTIPLE_ACTIVE_EMBEDDING_MODELS` | 500 |
| Python 서버 타임아웃/연결 실패 | `EMBEDDING_SERVER_UNAVAILABLE` | 503 |
| 응답 차원 불일치 | `EMBEDDING_DIMENSION_MISMATCH` | 500 |
| 빈 검색어 | `@NotBlank` 검증 (Controller 레이어, Issue 5에서 처리) | 400 |

---

## 설계 결정

**`EmbedResult` 내부 record 분리**
`QueryEmbeddingService`가 `(model, vector)`를 함께 반환하도록 설계했다.
Issue 5에서 `SearchQueryCommandService.createProcessing()`을 호출할 때 model과 vector를 분리 전달하지 않아도 되므로 조립 코드가 단순해진다.

**`markSuccess` / `markFailed`에 명시적 save 없음**
`SearchQueryCommandService`는 클래스 레벨 `@Transactional`이고, `SearchQuery`는 이미 영속 상태이므로 dirty checking으로 충분하다.
신규 엔티티 생성(`createProcessing`)에만 `save()`를 사용한다.

**503 격리 원칙**
Python 서버 장애 시 `EMBEDDING_SERVER_UNAVAILABLE(503)`로 응답하되, Spring 애플리케이션 자체는 정상 동작을 유지한다.
`RestClientException`을 try-catch로 잡아 DocGridException으로 변환하는 방식으로 격리한다.

**`search_type = VECTOR` 고정**
MVP는 dense vector 검색만 지원한다. `KEYWORD`, `HYBRID`는 Issue 명세상 2단계 확장 예정이므로 현재는 상수로 고정한다.
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package com.opensource.docgrid.domain.embedding.dto;

import com.opensource.docgrid.domain.embedding.entity.EmbeddingModel;

public record EmbedResult(EmbeddingModel model, float[] vector) {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
package com.opensource.docgrid.domain.embedding.dto.request;

public record EmbedRequest(String text) {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
package com.opensource.docgrid.domain.embedding.dto.response;

public record EmbedServerResponse(float[] vector, int dimension) {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package com.opensource.docgrid.domain.embedding.service.query;

import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestClientException;
import org.springframework.web.client.RestClient;

import com.opensource.docgrid.domain.embedding.dto.EmbedResult;
import com.opensource.docgrid.domain.embedding.dto.request.EmbedRequest;
import com.opensource.docgrid.domain.embedding.dto.response.EmbedServerResponse;
import com.opensource.docgrid.domain.embedding.entity.EmbeddingModel;
import com.opensource.docgrid.global.exception.DocGridException;
import com.opensource.docgrid.global.exception.ErrorCode;

import lombok.extern.slf4j.Slf4j;

@Slf4j
@Service
public class QueryEmbeddingService {

private final EmbeddingModelQueryService embeddingModelQueryService;
private final RestClient restClient;

public QueryEmbeddingService(
EmbeddingModelQueryService embeddingModelQueryService,
@Qualifier("embeddingRestClient") RestClient restClient
) {
this.embeddingModelQueryService = embeddingModelQueryService;
this.restClient = restClient;
}

public EmbedResult embed(String text) {
EmbeddingModel activeModel = embeddingModelQueryService.getActiveModel();

EmbedServerResponse response;
try {
response = restClient.post()
.uri("/embed")
.body(new EmbedRequest(text))
.retrieve()
.body(EmbedServerResponse.class);
Comment on lines +32 to +41

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

빈 문자열 요청을 외부 서버 호출 전에 400으로 거부하세요.

현재 embed("") 또는 공백 문자열도 활성 모델 조회 후 /embed로 전송됩니다. 요구사항의 400 validation 계약을 충족하도록 서비스 진입 시점에 검증하고, 빈 입력 테스트도 추가해야 합니다.

수정 예시
 public EmbedResult embed(String text) {
+    if (text == null || text.isBlank()) {
+        throw new DocGridException(ErrorCode.INVALID_PARAMETER);
+    }
+
     EmbeddingModel activeModel = embeddingModelQueryService.getActiveModel();
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
public EmbedResult embed(String text) {
EmbeddingModel activeModel = embeddingModelQueryService.getActiveModel();
EmbedServerResponse response;
try {
response = restClient.post()
.uri("/embed")
.body(new EmbedRequest(text))
.retrieve()
.body(EmbedServerResponse.class);
public EmbedResult embed(String text) {
if (text == null || text.isBlank()) {
throw new DocGridException(ErrorCode.INVALID_PARAMETER);
}
EmbeddingModel activeModel = embeddingModelQueryService.getActiveModel();
EmbedServerResponse response;
try {
response = restClient.post()
.uri("/embed")
.body(new EmbedRequest(text))
.retrieve()
.body(EmbedServerResponse.class);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/main/java/com/opensource/docgrid/domain/embedding/service/query/QueryEmbeddingService.java`
around lines 32 - 41, Update QueryEmbeddingService.embed to validate text at
method entry and reject null, empty, or whitespace-only input with the existing
400 validation mechanism before calling getActiveModel or the /embed endpoint.
Add a test covering blank input and confirming no external embedding request is
made.

} catch (RestClientException e) {
log.error("임베딩 서버 호출 실패: {}", e.getMessage());
throw new DocGridException(ErrorCode.EMBEDDING_SERVER_UNAVAILABLE);
}

if (response == null
|| response.vector() == null
|| response.vector().length != activeModel.getDimension()
|| response.dimension() != activeModel.getDimension()) {
log.error("임베딩 차원 불일치: expected={}, actual={}",
activeModel.getDimension(), response == null ? "null" : response.dimension());
throw new DocGridException(ErrorCode.EMBEDDING_DIMENSION_MISMATCH);
}

return new EmbedResult(activeModel, response.vector());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,16 @@ public class SearchQuery extends BaseEntity {
@Column(name = "error_message", columnDefinition = "TEXT")
private String errorMessage;

public void updateToSuccess(int latencyMs) {
this.status = ResultStatus.SUCCESS;
this.latencyMs = latencyMs;
}

public void updateToFailed(String errorMessage) {
this.status = ResultStatus.FAILED;
this.errorMessage = errorMessage;
}
Comment on lines +105 to +113

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

SearchQuery 상태 전이를 제한하세요.

현재 메서드는 PROCESSING 여부를 확인하지 않아 SUCCESS → FAILED, FAILED → SUCCESS 같은 재전이를 허용합니다. 재시도나 중복 호출이 발생하면 상태와 latencyMs/errorMessage가 서로 다른 실행의 값으로 저장될 수 있습니다.

PROCESSING에서만 전이하도록 방어하거나, 중복 호출을 허용한다면 필드 덮어쓰기 규칙을 명시하고 해당 경계 테스트를 추가하세요.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/main/java/com/opensource/docgrid/domain/search/entity/SearchQuery.java`
around lines 105 - 113, Restrict the state transitions in
SearchQuery.updateToSuccess and SearchQuery.updateToFailed so they apply only
when the current status is PROCESSING; otherwise leave the status and associated
latencyMs/errorMessage unchanged. Add boundary tests covering repeated and
conflicting calls, including SUCCESS → FAILED and FAILED → SUCCESS.


@Builder
public SearchQuery(User user, DocumentCollection collection, String queryText,
EmbeddingModel queryEmbeddingModel, float[] queryVector, SearchType searchType, int topK,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
* 검색/RAG 응답 처리 결과 상태.
*/
public enum ResultStatus {
PROCESSING,
SUCCESS,
FAILED
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package com.opensource.docgrid.domain.search.repository;

import org.springframework.data.jpa.repository.JpaRepository;

import com.opensource.docgrid.domain.search.entity.SearchQuery;

public interface SearchQueryRepository extends JpaRepository<SearchQuery, Long> {
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package com.opensource.docgrid.domain.search.service.command;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import com.opensource.docgrid.domain.collection.entity.DocumentCollection;
import com.opensource.docgrid.domain.embedding.entity.EmbeddingModel;
import com.opensource.docgrid.domain.search.entity.SearchQuery;
import com.opensource.docgrid.domain.search.enums.ResultStatus;
import com.opensource.docgrid.domain.search.enums.SearchType;
import com.opensource.docgrid.domain.search.repository.SearchQueryRepository;
import com.opensource.docgrid.domain.user.entity.User;

import lombok.RequiredArgsConstructor;

@Transactional
@Service
@RequiredArgsConstructor
public class SearchQueryCommandService {

private final SearchQueryRepository searchQueryRepository;

public SearchQuery createProcessing(
User user,
DocumentCollection collection,
String queryText,
EmbeddingModel model,
float[] vector,
int topK
) {
SearchQuery searchQuery = SearchQuery.builder()
.user(user)
.collection(collection)
.queryText(queryText)
.queryEmbeddingModel(model)
.queryVector(vector)
.searchType(SearchType.VECTOR)
.topK(topK)
.status(ResultStatus.PROCESSING)
.build();
return searchQueryRepository.save(searchQuery);
}

public void markSuccess(SearchQuery searchQuery, int latencyMs) {
searchQuery.updateToSuccess(latencyMs);
}

public void markFailed(SearchQuery searchQuery, String errorMessage) {
searchQuery.updateToFailed(errorMessage);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package com.opensource.docgrid.global.config;

import java.net.http.HttpClient;
import java.time.Duration;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.client.JdkClientHttpRequestFactory;
import org.springframework.web.client.RestClient;

@Configuration
public class EmbeddingServerConfig {

@Value("${embedding.server.base-url}")
private String baseUrl;

@Bean("embeddingRestClient")
public RestClient embeddingRestClient() {
HttpClient httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(5))
.build();
JdkClientHttpRequestFactory requestFactory = new JdkClientHttpRequestFactory(httpClient);
requestFactory.setReadTimeout(Duration.ofSeconds(5));

return RestClient.builder()
.baseUrl(baseUrl)
.requestFactory(requestFactory)
.build();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,18 @@ public enum ErrorCode {
HttpStatus.INTERNAL_SERVER_ERROR,
"EMBEDDING-MODEL-002",
"사용 가능한 임베딩 모델이 여러 개 설정되어 있습니다."
),

// SEARCH
EMBEDDING_SERVER_UNAVAILABLE(
HttpStatus.SERVICE_UNAVAILABLE,
"SEARCH-001",
"임베딩 서버를 사용할 수 없습니다."
),
EMBEDDING_DIMENSION_MISMATCH(
HttpStatus.INTERNAL_SERVER_ERROR,
"SEARCH-002",
"임베딩 차원이 설정된 모델과 일치하지 않습니다."
);

private final HttpStatus httpStatus;
Expand Down
Loading