[Bug] App UI chatbot is not responding #13
Description
Priority
Undecided
OS type
Ubuntu
Hardware type
Xeon-SPR
Installation method
- Pull docker images from hub.docker.com
- Build docker images from source
Deploy method
- Docker compose
- Docker
- Kubernetes
- Helm
Running nodes
Single Node
What's the version?
v1.1
Description
when the sandbox is running, testing the chatbot with any prompt is not giving any response. The backend API return status code 200, but the streaming response is empty.
But if import the sample workflow, the chatbot is working
Reproduce steps
construct a simple workflow like this and execute the sandbox
Raw log
### app-backend logs
kubectl logs -n sandbox-497d23df-81a1-4f10-aeec-fff739324679 app-backend-dfcf98f5d-zslsq
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
warnings.warn(
[2024-12-04 08:51:45,200] [ INFO] - Base service - CORS is enabled.
[2024-12-04 08:51:45,201] [ INFO] - Base service - Setting up HTTP server
[2024-12-04 08:51:45,202] [ INFO] - Base service - Uvicorn server setup on port 8888
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
[2024-12-04 08:51:45,214] [ INFO] - Base service - HTTP server setup successful
after initialize appService
add_remote_service
node {'category': 'Controls', 'connected_from': [], 'connected_to': ['opea_service@embedding_tei_langchain_0'], 'dependent_services': None, 'hideOutput': None, 'id': 'chat_input_0', 'inMegaservice': False, 'inference_params': {}, 'name': 'chat_input', 'params': {}, 'version': 1}
node {'category': 'Embeddings', 'connected_from': ['chat_input_0'], 'connected_to': ['opea_service@llm_tgi_0'], 'dependent_services': {'tei': {'endpoint': 'tei-0', 'port': 2081, 'huggingFaceToken': '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', 'modelName': 'BAAI/bge-large-en-v1.5'}}, 'hideOutput': None, 'id': 'opea_service@embedding_tei_langchain_0', 'inMegaservice': True, 'inference_params': {}, 'name': 'opea_service@embedding_tei_langchain', 'params': {}, 'version': 1}
adding Node opea_service@embedding_tei_langchain_0
node {'category': 'LLM', 'connected_from': ['opea_service@embedding_tei_langchain_0'], 'connected_to': ['chat_completion_0'], 'dependent_services': {'tgi': {'endpoint': 'tgi-0', 'port': 3081, 'huggingFaceToken': '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', 'modelName': 'Intel/neural-chat-7b-v3-3'}}, 'hideOutput': None, 'id': 'opea_service@llm_tgi_0', 'inMegaservice': True, 'inference_params': {'chat_template': '### You are a helpful, respectful and honest assistant to help the user with questions.\n### Context: {context}\n### Question: {question}\n### Answer:', 'frequency_penalty': 0, 'max_tokens': 17.0, 'presence_penalty': 1.03, 'streaming': 'True', 'temperature': 0.01, 'top_k': 10.0, 'top_p': 0.95, 'typical_p': 0.95}, 'name': 'opea_service@llm_tgi', 'params': {}, 'version': 1}
adding Node opea_service@llm_tgi_0
node {'category': 'Controls', 'connected_from': ['opea_service@llm_tgi_0'], 'connected_to': [], 'dependent_services': None, 'hideOutput': True, 'id': 'chat_completion_0', 'inMegaservice': False, 'inference_params': {}, 'name': 'chat_completion', 'params': {}, 'version': 1}
INFO: 10.244.0.15:39290 - "GET /metrics HTTP/1.1" 200 OK
INFO: 10.244.0.15:50504 - "GET /metrics HTTP/1.1" 200 OK
INFO: 10.244.0.15:39018 - "GET /metrics HTTP/1.1" 200 OK
====== handle_request ======
{'messages': [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': 'what is opea?'}], 'model': 'Intel/neural-chat-7b-v3-3', 'max_tokens': 50, 'temperature': 0.3, 'stream': True}
param_keys ['_abc_impl', '_is_protocol', '_proto_type_name', 'chat_template', 'frequency_penalty', 'id', 'language', 'max_new_tokens', 'max_tokens', 'model_computed_fields', 'model_config', 'model_extra', 'model_fields', 'model_fields_set', 'presence_penalty', 'repetition_penalty', 'streaming', 'temperature', 'top_k', 'top_p', 'typical_p']
align_inputs
align_inputs
inputs {'id': '974f9cd01000ec76367c963915e509aa', 'text': 'You are a helpful assistant.\nuser: what is opea?\n', 'embedding': [0.0014842215, -0.0007874283, -0.049801208, 0.008258938, -0.007281534, -0.018621862, 0.02535583, 0.018820103, 0.040295675, 0.0057557235, 0.00055266835, 0.032563105, -0.028629417, -0.01764253, -0.03405528, -0.015764663, -0.0065377853, -0.005178438, -0.030012824, -0.013302217, 0.008342498, 0.03203215, -0.022703506, -0.048085526, -0.04024585, -0.009712235, -0.017058542, -0.027343983, 0.060129505, 0.03904012, -0.055395335, 0.013349832, -0.008876224, -0.028308192, -0.03274076, -0.018631175, 0.06574453, -0.016380375, -0.033563077, -0.08100971, -0.0076630116, 0.040484082, 0.03513317, -0.04740087, -0.057180606, -0.00094860094, -0.0036328458, -0.012276605, 0.039696306, -0.048470076, 0.00097510533, 0.0118195275, 0.01376788, -0.03601918, -0.027363338, -0.05804714, 0.00078188383, 0.000606263, 0.0020469262, -0.008200143, 0.0020652458, 0.007732596, -0.040077504, -0.060761932, -0.005576274, 0.020836066, -0.024410497, -0.015478038, 0.010770301, 0.014985561, -0.018289624, 0.008128358, 0.014952776, 0.014834461, 0.0054370565, 0.02412881, 0.022155281, -0.018078698, 0.008154267, 0.031370655, -0.018443288, 0.031026853, 0.030320203, 0.015999326, -0.0042381594, 0.0058804257, 0.048885286, 0.045037176, 0.0383454, -0.010817231, -0.00091879995, 0.032703836, -0.012062053, 0.016199078, 0.030399645, 0.06277744, -0.025878578, 0.007919566, 0.055882443, -0.0024504443, 0.032455802, 0.018701961, 0.017415078, 0.030009784, -0.039880857, 0.03767283, 0.044450086, -0.036535963, 0.00437576, -0.02757716, 0.01367985, 0.031634066, -0.0035205705, -0.007870325, -0.03411071, 0.061477594, -0.0064570867, -0.0147709735, -0.02886108, 0.009557452, 0.0068778647, 0.02229676, -0.020852692, -0.015047334, 0.020768058, -0.04125316, -0.015840199, 0.03626474, -0.015186425, 0.009474164, -0.025835305, -0.031655803, 0.053013682, 0.02654124, 0.0041124285, 0.03634715, 0.010929981, 0.005266449, 0.050026022, -0.012798578, 0.02052257, -0.006012373, -0.0049816505, 0.0673207, 0.023769869, 0.0128544755, -0.010478309, -0.009865694, -0.03515629, 0.063027725, 0.030872265, 0.029319575, -0.041899294, 0.027337044, -0.008217014, -0.021071913, 0.007639699, -0.0039069564, 0.023305438, 0.04663796, -0.004286628, 0.0017195396, -0.025277646, -0.0048506726, -0.0405564, 0.017938923, 0.0021451034, 0.019605247, 3.3755794e-05, 0.02575452, 0.020288294, 0.028040703, -0.073280364, -0.032126147, 0.024973225, 0.006673149, 0.013209837, 0.04183229, 0.028214764, 0.029264614, -0.04907492, 0.012624637, -0.03883997, 0.019586772, 0.0077600186, 0.0121094035, 0.000797848, 0.011307659, -0.0058791596, -0.040446483, 0.026955348, 0.040811982, 0.010042056, -0.0053463583, -0.014337105, 0.046281457, -0.0121700065, 0.051421348, 0.0100891, -0.04349206, -0.04229191, 0.020010537, -0.00784978, 0.04497466, -0.018883327, -0.025092144, -0.027963229, -0.006458639, -0.0065602604, -0.017204413, 0.06288169, 0.06305618, -0.07030521, -0.02780095, 0.008984863, -0.005445256, -0.03184695, 0.07826042, -0.023503957, 0.045793895, -0.019513957, 0.040819593, -0.009947001, 0.024235373, -0.028977748, -0.02909195, -0.03757223, 0.02236877, -0.034925666, 0.008099268, -0.001040898, 0.023096502, -0.010738866, 0.055988863, 0.053455766, 0.025518192, 0.046227813, -0.005721153, -0.017473022, -0.032173242, 0.0060705133, 0.050454386, 0.05483364, 0.028191851, -0.024759708, -0.014101496, 0.04355632, 0.020844756, 0.0071889875, 0.024523996, -0.010393399, 0.04881234, 0.027480602, 0.012800857, -0.015806232, -0.014205999, 0.050039675, 0.046405226, -0.06286271, -0.062532455, -0.0018642668, 0.008055813, -0.005477472, 0.020848695, -0.0038140302, -0.03246459, 0.0013646098, 0.02144211, -0.032554068, -0.048862617, -0.0034485932, 0.0025622912, -0.016310643, 0.038892508, -0.038346883, -0.015234888, 0.00263745, -0.033928085, 0.015530378, -0.042655125, -0.025871316, -0.017659612, -0.06641021, 0.05252488, 0.01666058, 0.0067232293, -0.071841806, -0.027825478, 0.006992478, 0.0043213926, -0.02556621, -0.004341067, -0.029682402, -0.05138887, 0.036044862, -0.006153614, -0.011129817, 0.019010322, -0.053207826, -0.056534324, -0.06436549, -0.0034060453, -0.017952839, 0.0061712195, 0.006274613, 0.02910567, 0.039479647, 0.009115074, 0.029307125, 0.017482297, -0.005098505, 0.043493956, 0.012739074, 0.005109666, -0.028756816, 0.036810838, 0.049584035, -0.049941957, -0.012111699, 0.019773152, -0.034715872, -0.019423084, -0.028685413, 0.009380836, -0.016910693, -0.00071709225, -0.01702528, -0.09775458, 0.051229887, -0.024486009, 0.00626777, -0.019480906, 0.0081049455, 0.017662982, -0.0066036703, 0.032391902, -0.010816177, -0.008284581, -0.027245607, -0.009983272, 0.020735558, -0.020721782, 0.016901014, 0.011943259, -0.024469791, 0.003702343, 0.023133578, 0.00056499353, 0.00038998443, 0.015282666, -0.029173011, 0.0087846005, -0.037887126, -0.013036898, 0.054843318, 0.034073304, -0.024630113, -0.028888, -0.017821668, 0.02969791, 0.00062049413, -0.016389513, 0.029756555, -0.015297751, 0.0063304673, -0.0062314523, 0.010655743, 0.027892005, 0.049649723, -0.031269137, 0.023401918, -0.021417085, -0.04065951, 0.0018043035, -0.046238128, -0.025081713, 0.055870965, -0.035764225, 0.028332373, -0.022332119, -0.013215037, 0.00046399183, 0.004020001, 0.05689615, -0.07040199, -0.024101615, -0.033687256, 0.035278913, -0.035109363, 0.013986967, -0.022152025, -0.018317234, -0.010476284, -0.0030001414, -0.07451234, -0.03596879, 0.052168593, 0.003987029, 0.029847814, 0.004867846, 0.07358805, 0.018855678, 0.01958116, 0.05640551, -0.017039903, 0.010262962, -0.038307413, 0.029989306, 0.04027209, -0.019373568, 0.0014499213, 0.01745806, -0.032943204, 0.019120393, 0.029418856, -0.017113205, -0.010476937, 0.012255823, 0.01542855, 0.04418909, -0.042664587, -0.037068814, -0.04019115, 0.07121794, 0.008287089, 0.015349987, -0.018416215, -0.012623155, 0.040253248, 0.01933191, -0.04358035, -0.03180303, 0.01390167, -0.012744034, -0.006849919, 0.010118327, 0.030310035, -0.016260773, 0.00885727, -0.011545559, 0.04597883, 0.008082516, 0.01995461, -0.04353599, -0.05330756, 2.0384134e-05, -0.0017273832, 0.0048045493, -0.00971677, -0.041563362, 0.039524432, -0.024768861, 0.0028523265, 0.0024946684, 0.015313094, 0.005760046, 0.02693554, 0.03338987, 0.018192235, -0.012373079, 0.014066261, 0.013012573, 0.032027204, 0.031058544, -0.017646275, 0.031389534, 0.0073243026, 0.0017033758, 0.0070048436, 0.061049476, -0.035173245, 0.021703249, -0.005535854, 0.0008825124, 0.018587826, -0.03752732, -0.039566796, 0.008989466, -0.0073745954, 0.03133412, -0.021756193, 0.004956308, 0.05388736, -0.023230318, -0.027678661, -0.0464718, 0.065441005, 0.01310124, -0.052882083, 0.0015433755, -0.010130031, -0.0039613154, 0.009722425, -0.028414922, -0.022677433, -0.020950004, -0.01976778, -0.0007291909, -0.009904565, -0.00024027006, 0.02306378, -0.018317271, -0.0007162531, -0.047929585, -0.024764637, -0.04193519, -0.016244175, 0.012949498, 0.001636172, -0.012114834, -0.056198552, 0.033510108, -0.044076897, 0.030013103, 0.07550225, -0.010152587, 0.019848088, -0.017261392, -0.011306827, 0.02205606, -0.021572234, -0.027498977, -0.00934172, 0.028368568, 0.03529378, 0.0019233229, -0.008906409, 0.0027854205, -0.030913487, -0.093877055, 0.008919649, -0.01776935, -0.08737266, -0.042466253, -0.021655941, -0.004740576, 0.026281089, 0.0320908, -0.056199662, -0.0072861444, -0.033903286, -0.0066609685, -0.045771312, 0.0043850555, -0.04142283, 0.02554289, 0.012195581, 0.03419496, 0.004672809, -0.0035083327, 0.0054781614, 0.012197839, 0.020307437, -0.015420372, -0.018156951, -0.039511096, -0.042768396, 0.06046683, 0.019596752, 0.029080795, -0.0691263, 0.02853581, -0.0139195025, -0.006424014, -0.00016347169, -0.008534033, -0.06259341, 0.014391607, 0.033447944, -0.00069107814, -0.025763027, -0.021445896, 0.014202427, 0.020233482, 0.0011732448, -0.04080181, -0.030753521, -0.034554284, -0.02797305, -0.013350046, -0.007847133, -0.0046804515, -0.03580613, -0.010749642, 0.01704858, -0.038092855, 0.0038754286, 0.061214056, 0.051716678, -0.003782453, -0.0416052, 0.036699146, 0.025198223, 0.009770779, -0.014297889, -0.002757341, -0.0215329, -0.06642279, -0.01617025, -0.050455224, -0.023609044, 0.006814984, 0.020874334, -0.004315178, 0.00825154, 0.01881022, -0.035672996, 0.0044633006, 0.062338278, 0.0027949805, 0.010547862, 0.046335876, 0.018359188, -0.044096448, -0.024450399, -0.031148672, 0.0131209, -0.0041532735, 0.02658897, 0.05498339, -0.045135144, 0.047061197, 0.047909155, -0.019013537, -0.034219503, 0.017037971, 0.001273489, -0.025225952, 0.014452144, 0.02176346, 0.01954239, 0.008523058, -0.0023023784, 0.010849907, -0.02451086, 0.034045536, -0.02139401, 0.059425343, -0.04654187, -0.004462377, 0.03341046, -0.005514075, 0.013126785, -0.045914523, 0.0208726, 0.0046298485, 0.082091965, 0.041643728, 0.011904551, -0.038659185, 0.015724817, 0.005087987, 0.048020136, -0.04212827, -0.010723352, 0.04607156, -0.016601192, -0.026628813, -0.003318436, -0.00045425052, -0.029702183, 0.011828807, -0.020487849, -0.00042571125, 0.05844278, 0.04951891, -0.012757337, 0.0037219715, -0.020307407, -0.034981105, -0.041059602, -0.026969459, -0.0068591833, 0.010976311, -0.016612692, 0.004960881, -0.009568017, -0.006970961, -0.04119613, -0.0042783055, -0.0143015515, -0.021656714, 0.03270898, -0.014828148, -0.06044603, 0.0009143388, 0.004491908, -0.002984774, -0.041048974, -0.013115618, -0.014963055, 0.014486105, 0.075732335, -0.034364734, -0.0046602474, -0.07978562, 0.008475135, -0.018463029, 0.037122656, -0.022964837, 0.00092984666, 0.007573439, -0.004091717, 0.04142682, -0.012818723, 0.0015194413, -0.002818259, -0.015517273, 0.01816986, -0.025145117, -0.02605022, -0.041228436, -0.020077, -0.0017217282, -0.01566416, -0.012373298, -0.03759122, 0.017419584, 0.049119554, -0.00012947245, 0.026892187, -0.008801896, 0.0072158165, -0.014222186, -0.008567727, 0.019391261, 0.00079221633, 0.019305207, -0.006832762, 0.026111845, 0.0012805284, 0.022003958, 0.014483258, -0.027906539, -0.043621253, -0.043564275, -0.07410787, -0.0782983, -0.0068187467, 0.018683825, 0.036445267, 0.027690249, -0.035805035, -0.021805897, -0.0057595004, -0.031877995, -0.0037992874, -0.092393406, -0.038673867, 0.015836377, -0.010858985, -0.0058705015, 0.009274335, -0.0026330855, 0.003280085, 0.00058568764, 0.019776348, -0.02512082, -0.0011761351, 0.025833504, 0.003040061, -0.03963585, -0.011686949, 0.0010861026, 0.023278302, 0.012988519, -0.07324166, 0.00081428594, -0.014333758, 0.032782663, -0.0090671135, 0.026061809, 0.017442785, 0.032233167, 0.012516834, 0.03429188, 0.005993781, -0.02982568, 0.026106901, -0.03614318, 0.08401532, -0.014148758, 0.016624896, 0.0005091863, 0.022578271, -0.02959585, 0.015721707, 0.052306674, -0.015276944, 0.025192551, 0.009530034, 0.02860764, -0.018330636, -0.014580411, 0.01572902, 0.034126595, 0.037953947, -0.0016484899, -0.01315422, 0.006935639, 0.023419602, -0.050073244, 0.029478207, 0.031746622, 0.016811043, 0.009215483, -0.014029956, 0.015324866, 0.0062604705, -0.0020825255, 0.0015901435, 0.016469032, -0.062956125, -0.024414292, -0.057557702, -0.0346475, 0.039933152, -0.0062949182, 0.014507208, 0.0325031, -0.026863407, -0.0051340424, 0.01861936, 0.011233868, -0.062761925, 0.008500245, 0.023883319, -0.0031025123, 0.00585406, 0.034517206, 0.01950988, -0.029338917, -0.02022037, 0.01510705, -0.0170091, -0.0033646442, -0.017421141, -0.011118339, 0.027425166, 0.037337005, -0.0036770888, 0.056169946, -0.04752341, 0.015940037, -0.025329161, 0.03342517, -0.0547701, 0.022894138, 0.03725183, 0.0015720376, -0.03187732, 0.022018714, 0.017496299, 0.02952871, 0.034597438, 0.027425067, -0.011650893, 0.03648547, 0.028555283, -0.056623567, -0.04884793, -0.034599774, -0.01721811, -0.017983524, -0.021676831, -0.042457268, -0.041432403, 0.030886486, -0.013573248, -0.032203633, 0.034152422, 0.015408234, -0.024304532, -0.016223593, 0.055275306, -0.014439549, 0.025789076, -0.015911406, -0.033246957, 0.024662279, -0.06653291, 0.017616594, -0.045734633, 0.029670605, -0.019061921, 0.026672574, 0.014252961, -0.038991995, -0.039498616, -0.04035533, -0.011865421, 0.04840557, -0.012378851, -0.058934797, -0.014025106, 0.0005560315, -0.018932695, -0.00882772, -0.009278622, 0.028115118, 0.04172281, 0.06991235, 0.001036408, 0.04929175, 0.018305117, 0.011347029, -0.007428194, -0.00405036, -0.0069229277, -0.024096923, -0.012319021, -0.012460656, 0.027414339, -0.027296433, -0.019985797, 0.035107892, 0.042024396, -0.0024047783, -0.02972621, 0.017794194, -0.030317167, -0.04149981, -0.009488276, -0.0036072126, 0.05606502, 0.023215216, 0.038593512, 0.01349359, -0.08312135, 0.22318274, 0.05273501, -0.0019223862, 0.08220261, 0.010101311, 0.050735965, 0.04240991, -0.01048379, -0.030306665, -0.044170186, 0.024599377, -0.0043506855, 0.01842783, 0.0083399825, -0.004093967, 0.030552177, -0.015808066, 0.005837579, 0.020659164, -0.020252954, -0.04084612, -0.021376755, -0.002526943, 0.045244332, -0.014859966, 0.022259735, 0.040816884, -0.030448738, -0.0025315224, -0.05005978, 0.046295315, -0.035346214, 0.04312708, -0.008566909, -0.0261359, 0.0063484595, -0.014808388, -0.08077185, -0.025598455, 0.023333762, 0.022939974, -0.02123092, -0.053016182, 0.01283913, -0.010103003, 0.04944733, -0.021619333, 0.004282004, 0.04606725, -0.031665526, 0.033997376, -0.049593564, 0.04246837, 0.020359002, -0.026401643, 5.5085824e-05, 0.04968067, 0.0027685754, 0.0038670956, 0.013469084, 0.033544004, -0.0053905863, -0.0039421883, 0.0064337132, -0.0021070258, -0.0070055197, 0.04049458, 0.0768693, 0.0035096533, 0.0025290372, -0.011376137, -0.018450068, 0.014245208, -0.023168152, 0.07374461, 0.0041268216, 0.037051547, 0.01984258, -0.0031269575, -0.06851008, 0.013955721, -0.00048409845, -0.03838715, 0.008731608, 0.0013947258, 0.025859984, 0.0039963783, -0.014689224, -0.05590672, 0.006522296, 0.030044299, 0.020133963, -0.025298102, -0.009450078, 0.021324277],
'search_type': 'similarity', 'k': 4, 'distance_threshold': None, 'fetch_k': 20, 'lambda_mult': 0.5, 'score_threshold': 0.2, 'constraints': None, 'max_tokens': 50, 'max_new_tokens': 50, 'top_k': 10.0, 'top_p': 0.95, 'typical_p': 0.95, 'temperature': 0.3, 'frequency_penalty': 0, 'presence_penalty': 1.03, 'repetition_penalty': 1.03, 'streaming': 'True', 'language': 'auto', 'chat_template': '### You are a helpful, respectful and honest assistant to help the user with questions.\n### Context: {context}\n### Question: {question}\n### Answer:'}
runtime_graph OrderedDict([('opea_service@embedding_tei_langchain_0', {'opea_service@llm_tgi_0'}), ('opea_service@llm_tgi_0', set())])
INFO: 10.244.0.8:51524 - "POST /v1/app-backend?ns=sandbox-497d23df-81a1-4f10-aeec-fff739324679 HTTP/1.1" 200 OK
INFO: 10.244.0.15:59652 - "GET /metrics HTTP/1.1" 200 OK
### llm-tgi logs
INFO: 10.244.0.194:33654 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity