Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Evaluation] ./tester 테스트 케이스 #249

Open
yechoi42 opened this issue Apr 3, 2021 · 29 comments
Open

[Evaluation] ./tester 테스트 케이스 #249

yechoi42 opened this issue Apr 3, 2021 · 29 comments

Comments

@yechoi42
Copy link
Member

yechoi42 commented Apr 3, 2021

Test GET http://localhost:1234/
Test POST http://localhost:1234/ with a size of 0
Test HEAD http://localhost:1234/
Test GET http://localhost:1234/directory
Test GET http://localhost:1234/directory/youpi.bad_extension
Test GET http://localhost:1234/directory/youpi.bla
Test GET Expected 404 on http://localhost:1234/directory/oulalala
Test GET http://localhost:1234/directory/nop
Test GET http://localhost:1234/directory/nop/
Test GET http://localhost:1234/directory/nop/other.pouic
Test GET Expected 404 on http://localhost:1234/directory/nop/other.pouac
Test GET Expected 404 on http://localhost:1234/directory/Yeah
Test GET http://localhost:1234/directory/Yeah/not_happy.bad_extension
Test Put http://localhost:1234/put_test/file_should_exist_after with a size of 1000
Test Put http://localhost:1234/put_test/file_should_exist_after with a size of 10000000
Test POST http://localhost:1234/directory/youpi.bla with a size of 100000000
Test POST http://localhost:1234/directory/youpla.bla with a size of 100000000
Test POST http://localhost:1234/directory/youpi.bla with a size of 100000 with special headers
Test POST http://localhost:1234/post_body with a size of 0
Test POST http://localhost:1234/post_body with a size of 100
Test POST http://localhost:1234/post_body with a size of 200
Test POST http://localhost:1234/post_body with a size of 101
Test multiple workers(5) doing multiple times(15): GET on /
Test multiple workers(20) doing multiple times(5000): GET on /
Test multiple workers(128) doing multiple times(50): GET on /directory/nop
Test multiple workers(20) doing multiple times(5): Put on /put_test/multiple_same with size 1000000
Test multiple workers(20) doing multiple times(5): Post on /directory/youpi.bla with size 100000000
@yechoi42
Copy link
Member Author

yechoi42 commented Apr 3, 2021

Test POST http://localhost:1234/directory/youpi.bla with a size of 100000 with special headers

HTTP_X_SECRET_HEADER_FOR_TEST should be the env var

@hochan222
Copy link
Member

hochan222 commented Apr 7, 2021

Report

image

  • Result

image


other issue occurs => retest

@hochan222
Copy link
Member

hochan222 commented Apr 7, 2021

HTTP_X_SECRET_HEADER_FOR_TEST defines response message

@hochan222
Copy link
Member

image

success

@hochan222
Copy link
Member

hochan222 commented Apr 7, 2021

success 마지막 woker만 남음

@hochan222
Copy link
Member

hochan222 commented Apr 7, 2021

Accept OK
Accept OK
Accept OK
Accept OK
Accept OK
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::1:: 16 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::1:: 17 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::1:: 18 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::1:: 19 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::4::
OOOOOOO END 3938
::4::
OOOOOOO OK 3938
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::1:: 16 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::4::
OOOOOOO END 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO OK 3938
::4::
OOOOOOO END 3938
::4::
OOOOOOO OK 3938
::1:: 9 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
GET MESSAGE SUCCESS 
RESETREQUEST) BODY SIZE: 0
STATUS CODE: 200
SET WRITE FD :D
::4::
OOOOOOO END 3938
::1:: 16 ::
GET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzipGET / HTTP/1.1
Host: localhost:8080
User-Agent: Go-http-client/1.1
Accept-Encoding: gzip


CASE 3
select error
9
4
22
12
9

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

feature/multirequest

Test multiple workers(5) doing multiple times(15): GET on /
통과

Test multiple workers(20) doing multiple times(5000): GET on /
실패
run.sh: line 55: 2965 Killed: 9 ./Webserv config/${file_name}

@hochan222
Copy link
Member

와... ㅠㅠㅠ

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

20 workers

모두 SYN_SENT인데 accept이 안된 걸까요?

yechoi@yechoiui-MacBookPro webserv % bash killport.sh 
COMMAND  PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
tester  3215 yechoi    7u  IPv6 0x14c731459cecfea5      0t0  TCP localhost:52202->localhost:http-alt (SYN_SENT)
tester  3215 yechoi    9u  IPv4 0x14c731459e450795      0t0  TCP localhost:52200->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   10u  IPv4 0x14c731459e44cc55      0t0  TCP localhost:52201->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   11u  IPv6 0x14c731459ced1725      0t0  TCP localhost:52204->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   13u  IPv6 0x14c731459ced2365      0t0  TCP localhost:52203->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   14u  IPv6 0x14c731459cec5ea5      0t0  TCP localhost:52206->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   15u  IPv6 0x14c731459cec7105      0t0  TCP localhost:52205->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   16u  IPv6 0x14c731459cec6ae5      0t0  TCP localhost:52208->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   17u  IPv6 0x14c731459ce65205      0t0  TCP localhost:51951->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   18u  IPv6 0x14c731459ce65825      0t0  TCP localhost:51954->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   19u  IPv6 0x14c731459ce60ea5      0t0  TCP localhost:51952->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   20u  IPv6 0x14c731459ce61ae5      0t0  TCP localhost:51953->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   21u  IPv6 0x14c731459ce614c5      0t0  TCP localhost:51956->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   22u  IPv6 0x14c731459ce62725      0t0  TCP localhost:51955->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   23u  IPv6 0x14c731459ce64be5      0t0  TCP localhost:51957->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   24u  IPv6 0x14c731459cec95c5      0t0  TCP localhost:52210->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   26u  IPv6 0x14c731459cec8365      0t0  TCP localhost:52207->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   28u  IPv6 0x14c731459cec9be5      0t0  TCP localhost:52209->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   29u  IPv6 0x14c731459ceca205      0t0  TCP localhost:52211->localhost:http-alt (SYN_SENT)
tester  3215 yechoi   30u  IPv4 0x14c731459e44c275      0t0  TCP localhost:52212->localhost:http-alt (SYN_SENT)

@hochan222
Copy link
Member

hochan222 commented Apr 11, 2021

네 established가 되어야 연결된거에요 그전에 클라이언트에서 synsent로 기다립니다.

@hochan222
Copy link
Member

혹시 tester말고 webserv상태도 보여주실수있나요?

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

그건 어떻게 확인하나요???

웹서브는 아래 메세지와 함께 스스로 죽었어요
run.sh: line 55: 2965 Killed: 9 ./Webserv config/${file_name}

아래처럼 계속 읽어오려고 시도하다 죽어요. GET MESSSAGE SUCCESS 란 문구가 안나온 걸로 봐서는, getMessage 리턴값이 SUCCESSS는 아니네요.

::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::
::1:: 5 ::

일단 제가 짐작하는 문제 원인은

  • socket 관리 실패
  • 혹시 메세지가 잘려서 오는지...? 그래서 recv(msg_peek)만 무한정하다가 read는 못하고 죽어버리는 상황이 생기는지?

@hochan222
Copy link
Member

hochan222 commented Apr 11, 2021

앗앗 죽어서 사라졌나봐요. 원래는 아래 바로나옵니다. 그러면 SENT_SYN이 나오는 이유는 accept되기전에 Webserv 가 중단돼서인것같습니다.

Sock 관리실패면 select err 9가 떳을텐데 음..

이게 동일 문제인지는 모르겠으나 지금 계속 검색중인데 close()만으로 소켓이 끊어졌다고 확신할수없는것같습니다. ( close가 바로 버퍼까지 비워주는건아닌것같습니다)

저도 온전치 못한 메세지때문인것같기도합니다 음음.. 뭔가 끊어주는 기준을 정해야할것같아요 로그는 어떤가요?

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

Test multiple workers(5) doing multiple times(15): GET on /

Test multiple workers(20) doing multiple times(5000): GET on /

두개가 같은 요청이라서 구분을 할 수가 없네요 ㅠㅠ 이게 5워커의 요청인지 20워커의 요청인지, 20워커의 요청이 들어오기나 한 것인지... 제 생각엔 들어오지도 않았을 것 같습니다

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

그렇네요... 왜 죽어버리지 .... 아무튼 읽지도 못한채로 죽어버리는 거니까... 반짤려서 온다고 생기는 문제는 아니겠네요.

run.sh: line 55: 3665 Killed: 9 ./Webserv config/${file_name}

이런 메세지 나오면서 죽잖아요. 혹시 3665 이게 pid 같은데, webserv의 부모 pid겠죠? cgi pid가 죽는다고 프로그램이 꺼지진 않을테니.. .

그리고 앞선 메세지에서 저희 요청이 정상처리되면 close 안하기 때문에 close 하는 부분은 지금은 없는 것 같아요. 아니면 tester의 close 문제 얘기하는 거에요?

@hochan222
Copy link
Member

hochan222 commented Apr 11, 2021

생각해보니 SENT_SYN 가 서로 연결되기 전 상태니까 read peek문제가 있더라도 그 전에 발생한 문제겠네요..

http://hyacinth.byus.net/moniwiki/wiki.php/TCP%20%EC%97%B0%EA%B2%B0%20%EC%83%81%ED%83%9C%20%EC%9D%98%EB%AF%B8

@hochan222
Copy link
Member

hochan222 commented Apr 11, 2021

저희쪽 close에관한 것이었는데 생각해보니 그 문제는 아닌것같아요 생각 조금 더 해보겠습니다.

@yechoi42
Copy link
Member Author

Accept OK
::1:: 17 ::
shold peek false
STEP2) READ
--- READ: Parse message fail
GET
----------------------------
GET MESSAGE FAIL
Accept OK
::1:: 28 ::
shold peek false
STEP2) READ
--- READ: Parse message fail
GET
----------------------------
GET MESSAGE FAIL
Accept OK
::1:: 10 ::
shold peek false
STEP2) READ
--- READ: Parse message fail
GET
----------------------------
GET MESSAGE FAIL
Accept OK
::1:: 29 ::
shold peek false
STEP2) READ
--- READ: Parse message fail
GET
----------------------------
GET MESSAGE FAIL

ret ==0 일때도 continue를 해줬던 게 가장 큰 문제였고 그건 fail 처리해줬습니다.
그 다음 문제는 위 처럼 로그를 찍어봤는데 메세지가 잘린 채로 들어왔는데 parse를 시도하는 게 문제입니다.
아래 부분이 파싱을 시도하려하는 메세지입니다.

--- READ: Parse message fail
GET
----------------------------

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

Test multiple workers(20) doing multiple times(5000): GET on /
성공

Test multiple workers(128) doing multiple times(50): GET on /directory/nop
성공

Test multiple workers(20) doing multiple times(5): Put on /put_test/multiple_same with size 1000000
성공

눈물이 난다...............ㅠㅠㅠㅠㅠㅠㅠㅠ흫아하앟악

@yechoi42
Copy link
Member Author

yechoi42 commented Apr 11, 2021

Test multiple workers(20) doing multiple times(5): Post on /directory/youpi.bla with size 100000000
시도중

이거 근데 저희 한번 할 때도 오래걸리거든요... 근데 총 100번 하니까.... 시간이 이게... 아마 두시간 걸릴듯;;; 개선이 필요하긴 하겠네요

@jwon42
Copy link
Collaborator

jwon42 commented Apr 11, 2021

와우.............................ㅜㅡㅜ

@yechoi42
Copy link
Member Author

아무튼 앞에 것들의 주된 개선사항은

  • read 하기 전에 recv해서 read 할 양 정해줌
  • 그 과정에서 read 할 양을 정하는 것이나 파싱 조건이 조금씩 달라지거나 놓친 게 있는데 그런 것들을 손봐줌

@hochan222
Copy link
Member

hochan222 commented Apr 11, 2021

@yechoi42 @jwon42 와ㅠㅠㅠㅠㅠㅠㅠㅠㅠ 대박이에요 ㅠㅠㅠㅠㅠㅠㅠㅠㅠㅠ 와....>!!!! 와 ㅠㅠㅠㅠㅠㅠㅠㅠ 너무 고생하셨어요 ㅠㅠㅠㅠ 흐흐흐흐흐휴ㅠㅠㅠㅠㅠ

@hochan222
Copy link
Member

@yechoi42 @jwon42 제가 테스터기라도 기깔나게 만들어 놓겠습니다.

@yechoi42
Copy link
Member Author

Test multiple workers(20) doing multiple times(5): Post on /directory/youpi.bla with size 100000000


GG, So far so good! Run your own tests now! :D


@hochan222
Copy link
Member

ㅠㅠㅠㅠㅠㅠㅠ

@hochan222
Copy link
Member

while 1 ; do ls -al ; sleep 1 ; clear ; done

@yechoi42
Copy link
Member Author

오늘 시도해본 것

  • 파일을 다시 pipe fd로 바꿔보기 -> 여전히 EPIPE
	int parent_write = fd[1];
	int cgi_stdin = fd[0];
  • ::4:: 에서 write 대신 send 써보기 -> 여전히 EPIPE
  • CGI 자식 프로세스에서 dup2(clientfd, STDOUT)
FATAL ERROR ON LAST TEST: Post http://localhost:8080/directory/youpi.bla: net/http: HTTP/1.x transport connection broken: malformed HTTP version "Status:"

이렇게 하니까 직접 보내지고!!!! 대신 형식이 맞지 않아서 에러가 나옵니다. 암튼 직접 물려준다는 것의 의미는 이런 것이 아닐까... 그래서 내일 시도해볼 건, 일단 헤더만 정리해주고 그 다음부터 clientfd로 직접 쏴주게요.

@hochan222
Copy link
Member

hochan222 commented Apr 22, 2021

이렇게 하니까 직접 보내지고!!!!

@yechoi42 직접 보내지나 검증을 몇가지 해보았습니다.

검증

우선 검증을 위해 헤더는 동일하게 안보내주었습니다. 클라이언트로 테스트해보았는데 직접물린경우 아무것도 출력되지않습니다.

  • dup2(fd[0], STDIN_FILENO)

  • dup2(m_client_socket, STDIN_FILENO)의 경우

  • dup2(clientfd, STDIN_FILENO)의 경우

오류 추정

  • dup 방식에 문제가있다.
  • dup는 잘됐지만 pipe가 빨리 닫혔다

  • 검증에 사용된 python 코드
import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('localhost', 8080))
cmd = "POST /directory/YoupiBanane/aa.bla HTTP/1.1\r\nAccept: */*\r\nUser-Agent: rawtester\r\nHost: localhost:8080\r\nContent-Length: 7\r\n\r\nabcdefg"
mysock.send(cmd)
while True:
    data = mysock.recv(466)
    if(len(data) < 1):
        break
    print(data.decode())
mysock.close()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants