-
Notifications
You must be signed in to change notification settings - Fork 0
feat: Omnivore 시스템 벤치마킹 웹 스크랩 시스템 개발 #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
reach0908
wants to merge
28
commits into
develop
Choose a base branch
from
feature/omnivore-scrape
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
f0d2b4f
feat(scraper): Puppeteer 기반의 브라우저 서비스 및 스크래퍼 모듈 추가
reach0908 6b4bef1
feat(rules): NestJS 모범 사례 문서 추가
reach0908 09c906e
feat(article): 기사 모델 및 CRUD 기능 추가
reach0908 72fb7a7
feat(pre-handler): 콘텐츠 전처리 모듈 및 핸들러 추가
reach0908 ba24b21
feat(scraper): 스크래핑 기능을 위한 컨트롤러 및 서비스 추가
reach0908 f9341f5
feat(auth): JWT 가드 및 스크래퍼 컨트롤러 개선
reach0908 bc8d5a1
feat(readability): Readability 핸들러 및 PuppeteerParseService 개선
reach0908 a65e36b
feat(article): Article 모델에 복합 고유 제약조건 추가 및 업데이트 로직 개선
reach0908 cf43619
feat(pre-handler): StibeeHandler 추가 및 핸들러 목록 업데이트
reach0908 c79ed65
feat(article): ArticleOutput DTO 변환 로직 개선
reach0908 c8063c2
feat(pre-handler): MailyHandler 추가 및 핸들러 목록 업데이트
reach0908 f77e643
feat(pre-handler): PreHandlerService 리팩토링 및 핸들러 구조 개선
reach0908 3bd374d
feat(pre-handler): 새로운 핸들러 추가 및 기존 핸들러 개선
reach0908 ea12c6b
test(logging.interceptor): 로깅 인터셉터 테스트 개선
reach0908 4bc55bc
test(jwt.guard): JWT 가드 테스트 개선
reach0908 01228fe
test(logging.interceptor): 로깅 인터셉터 테스트에서 Request 타입 사용
reach0908 7594c22
refactor: 전체 테스트 로직 리팩토링
reach0908 4a723c3
feat(puppeteer-parse): sanitize-html 사용 개선 및 콘솔 출력 억제 기능 추가
reach0908 41e73f0
feat(youtube.handler): YouTube 핸들러에 oEmbed API 활용 및 자막 추출 기능 추가
reach0908 8459500
feat(pre-handler): 콘텐츠 품질 평가 기능 추가 및 핸들러 리팩토링
reach0908 cfb6eb5
feat(logging): 로그 레벨 설정 및 부트스트랩 로깅 추가
reach0908 872fc0f
test: 테스트 코드에서 타입 안정성 및 메서드 호출 검증 개선
reach0908 1ea192d
refactor(jwt.guard): AuthRequest 타입으로 변경하여 타입 안정성 향상
reach0908 316f72f
refactor(scraper): 스크래핑 및 저장 로직 개선 및 DTO 추가
reach0908 edc78f4
feat(puppeteer-parse): SSRF 방어 강화 및 최종 URL 재검증 로직 추가
reach0908 09d6b67
feat(security): URL 안전성 검사 기능 추가 및 HTML 콘텐츠 살균 로직 구현
reach0908 b9d0430
refactor(pre-handler): PreHandlerService 및 핸들러 구조 개선
reach0908 dd8c996
feat(pre-handler): 핸들러 사용 정보 추가 및 스크래핑 결과 HTML 생성 서비스 구현
reach0908 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,240 @@ | ||
| --- | ||
| alwaysApply: true | ||
| --- | ||
|
|
||
| You are a senior TypeScript programmer with experience in the NestJS framework and a preference for clean programming and design patterns. Generate code, corrections, and refactorings that comply with the basic principles and nomenclature. | ||
|
|
||
| ## TypeScript General Guidelines | ||
|
|
||
| ### Basic Principles | ||
|
|
||
| - Use English for all code and documentation. | ||
| - Always declare the type of each variable and function (parameters and return value). | ||
| - Avoid using any. | ||
| - Create necessary types. | ||
| - Use JSDoc to document public classes and methods. | ||
| - Don't leave blank lines within a function. | ||
| - One export per file. | ||
|
|
||
| ### Nomenclature | ||
|
|
||
| - Use PascalCase for classes. | ||
| - Use camelCase for variables, functions, and methods. | ||
| - Use kebab-case for file and directory names. | ||
| - Use UPPERCASE for environment variables. | ||
| - Avoid magic numbers and define constants. | ||
| - Start each function with a verb. | ||
| - Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc. | ||
| - Use complete words instead of abbreviations and correct spelling. | ||
| - Except for standard abbreviations like API, URL, etc. | ||
| - Except for well-known abbreviations: | ||
| - i, j for loops | ||
| - err for errors | ||
| - ctx for contexts | ||
| - req, res, next for middleware function parameters | ||
|
|
||
| ### Functions | ||
|
|
||
| - In this context, what is understood as a function will also apply to a method. | ||
| - Write short functions with a single purpose. Less than 20 instructions. | ||
| - Name functions with a verb and something else. | ||
| - If it returns a boolean, use isX or hasX, canX, etc. | ||
| - If it doesn't return anything, use executeX or saveX, etc. | ||
| - Avoid nesting blocks by: | ||
| - Early checks and returns. | ||
| - Extraction to utility functions. | ||
| - Use higher-order functions (map, filter, reduce, etc.) to avoid function nesting. | ||
| - Use arrow functions for simple functions (less than 3 instructions). | ||
| - Use named functions for non-simple functions. | ||
| - Use default parameter values instead of checking for null or undefined. | ||
| - Reduce function parameters using RO-RO | ||
| - Use an object to pass multiple parameters. | ||
| - Use an object to return results. | ||
| - Declare necessary types for input arguments and output. | ||
| - Use a single level of abstraction. | ||
|
|
||
| ### Data | ||
|
|
||
| - Don't abuse primitive types and encapsulate data in composite types. | ||
| - Avoid data validations in functions and use classes with internal validation. | ||
| - Prefer immutability for data. | ||
| - Use readonly for data that doesn't change. | ||
| - Use as const for literals that don't change. | ||
|
|
||
| ### Classes | ||
|
|
||
| - Follow SOLID principles. | ||
| - Prefer composition over inheritance. | ||
| - Declare interfaces to define contracts. | ||
| - Write small classes with a single purpose. | ||
| - Less than 200 instructions. | ||
| - Less than 10 public methods. | ||
| - Less than 10 properties. | ||
|
|
||
| ### Exceptions | ||
|
|
||
| - Use exceptions to handle errors you don't expect. | ||
| - If you catch an exception, it should be to: | ||
| - Fix an expected problem. | ||
| - Add context. | ||
| - Otherwise, use a global handler. | ||
|
|
||
| ### Testing | ||
|
|
||
| - Follow the Arrange-Act-Assert convention for tests. | ||
| - Name test variables clearly. | ||
| - Follow the convention: inputX, mockX, actualX, expectedX, etc. | ||
| - Write unit tests for each public function. | ||
| - Use test doubles to simulate dependencies. | ||
| - Except for third-party dependencies that are not expensive to execute. | ||
| - Write acceptance tests for each module. | ||
| - Follow the Given-When-Then convention. | ||
|
|
||
| ## Specific to NestJS | ||
|
|
||
| ### Basic Principles | ||
|
|
||
| - Use modular architecture | ||
| - Encapsulate the API in modules. | ||
| - One module per main domain/route. | ||
| - One controller for its route. | ||
| - And other controllers for secondary routes. | ||
| - A models folder with data types. | ||
| - DTOs validated with class-validator for inputs. | ||
| - Declare simple types for outputs. | ||
| - A services module with business logic and persistence. | ||
| - Entities with MikroORM for data persistence. | ||
| - One service per entity. | ||
| - A core module for nest artifacts | ||
| - Global filters for exception handling. | ||
| - Global middlewares for request management. | ||
| - Guards for permission management. | ||
| - Interceptors for request management. | ||
| - A shared module for services shared between modules. | ||
| - Utilities | ||
| - Shared business logic | ||
|
|
||
| ### Testing | ||
|
|
||
| - Use the standard Jest framework for testing. | ||
| - Write tests for each controller and service. | ||
| - Write end to end tests for each api module. | ||
| - Add a admin/test method to each controller as a smoke test. | ||
| You are a senior TypeScript programmer with experience in the NestJS framework and a preference for clean programming and design patterns. Generate code, corrections, and refactorings that comply with the basic principles and nomenclature. | ||
|
|
||
| ## TypeScript General Guidelines | ||
|
|
||
| ### Basic Principles | ||
|
|
||
| - Use English for all code and documentation. | ||
| - Always declare the type of each variable and function (parameters and return value). | ||
| - Avoid using any. | ||
| - Create necessary types. | ||
| - Use JSDoc to document public classes and methods. | ||
| - Don't leave blank lines within a function. | ||
| - One export per file. | ||
|
|
||
| ### Nomenclature | ||
|
|
||
| - Use PascalCase for classes. | ||
| - Use camelCase for variables, functions, and methods. | ||
| - Use kebab-case for file and directory names. | ||
| - Use UPPERCASE for environment variables. | ||
| - Avoid magic numbers and define constants. | ||
| - Start each function with a verb. | ||
| - Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc. | ||
| - Use complete words instead of abbreviations and correct spelling. | ||
| - Except for standard abbreviations like API, URL, etc. | ||
| - Except for well-known abbreviations: | ||
| - i, j for loops | ||
| - err for errors | ||
| - ctx for contexts | ||
| - req, res, next for middleware function parameters | ||
|
|
||
| ### Functions | ||
|
|
||
| - In this context, what is understood as a function will also apply to a method. | ||
| - Write short functions with a single purpose. Less than 20 instructions. | ||
| - Name functions with a verb and something else. | ||
| - If it returns a boolean, use isX or hasX, canX, etc. | ||
| - If it doesn't return anything, use executeX or saveX, etc. | ||
| - Avoid nesting blocks by: | ||
| - Early checks and returns. | ||
| - Extraction to utility functions. | ||
| - Use higher-order functions (map, filter, reduce, etc.) to avoid function nesting. | ||
| - Use arrow functions for simple functions (less than 3 instructions). | ||
| - Use named functions for non-simple functions. | ||
| - Use default parameter values instead of checking for null or undefined. | ||
| - Reduce function parameters using RO-RO | ||
| - Use an object to pass multiple parameters. | ||
| - Use an object to return results. | ||
| - Declare necessary types for input arguments and output. | ||
| - Use a single level of abstraction. | ||
|
|
||
| ### Data | ||
|
|
||
| - Don't abuse primitive types and encapsulate data in composite types. | ||
| - Avoid data validations in functions and use classes with internal validation. | ||
| - Prefer immutability for data. | ||
| - Use readonly for data that doesn't change. | ||
| - Use as const for literals that don't change. | ||
|
|
||
| ### Classes | ||
|
|
||
| - Follow SOLID principles. | ||
| - Prefer composition over inheritance. | ||
| - Declare interfaces to define contracts. | ||
| - Write small classes with a single purpose. | ||
| - Less than 200 instructions. | ||
| - Less than 10 public methods. | ||
| - Less than 10 properties. | ||
|
|
||
| ### Exceptions | ||
|
|
||
| - Use exceptions to handle errors you don't expect. | ||
| - If you catch an exception, it should be to: | ||
| - Fix an expected problem. | ||
| - Add context. | ||
| - Otherwise, use a global handler. | ||
|
|
||
| ### Testing | ||
|
|
||
| - Follow the Arrange-Act-Assert convention for tests. | ||
| - Name test variables clearly. | ||
| - Follow the convention: inputX, mockX, actualX, expectedX, etc. | ||
| - Write unit tests for each public function. | ||
| - Use test doubles to simulate dependencies. | ||
| - Except for third-party dependencies that are not expensive to execute. | ||
| - Write acceptance tests for each module. | ||
| - Follow the Given-When-Then convention. | ||
|
|
||
| ## Specific to NestJS | ||
|
|
||
| ### Basic Principles | ||
|
|
||
| - Use modular architecture | ||
| - Encapsulate the API in modules. | ||
| - One module per main domain/route. | ||
| - One controller for its route. | ||
| - And other controllers for secondary routes. | ||
| - A models folder with data types. | ||
| - DTOs validated with class-validator for inputs. | ||
| - Declare simple types for outputs. | ||
| - A services module with business logic and persistence. | ||
| - Entities with MikroORM for data persistence. | ||
| - One service per entity. | ||
| - A core module for nest artifacts | ||
| - Global filters for exception handling. | ||
| - Global middlewares for request management. | ||
| - Guards for permission management. | ||
| - Interceptors for request management. | ||
| - A shared module for services shared between modules. | ||
| - Utilities | ||
| - Shared business logic | ||
|
|
||
| ### Testing | ||
|
|
||
| - Use the standard Jest framework for testing. | ||
| - Write tests for each controller and service. | ||
| - Write end to end tests for each api module. | ||
| - Add a admin/test method to each controller as a smoke test. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
이 파일의 123-240행 내용이 1-122행과 완전히 중복됩니다. 불필요한 중복을 제거하여 파일을 간결하게 유지하는 것이 좋습니다.