"start index", "end index" doesn't match in n2c2 data

[sean0042]

Hi.

I am trying to use the DISCQ dataset but the index you provide in discq_questions_final.csv file doesn't seem to match with i2b2 2010 data (nor the original MIMIC-III notesevents).

First, I thought that every index was shifted to certain amount, but it seems to be not the case.
Below example is the first three rows from discq_questions_final.csv

I downloaded the data from n2c2 portal, (2010 relations)

[elehman16]

Can you try using the read_file function here:

discq/src/utils.py

Line 8 in 27ff842

def read_file(f_name: str, dir_path: str):

[thomasnguyen92]

this issue still exists.
For example, I run the following codes

discq_questions=pd.read_csv(discq_questions_final_path)
index_mappers={}
for index, row in discq_questions.iterrows():
    file_content,_=read_file(row['id'], relations_txt_path)
    discq_questions.at[index,'text']=file_content
    index_mappers[index]=_

for n in range(len(discq_questions)):
    a=discq_questions.iloc[n]['text'][index_mappers[n][discq_questions.iloc[n]['start_index']-1]:index_mappers[n][discq_questions.iloc[n]['end_index']]]
    b=discq_questions.iloc[n]['reasoning']
    assert b in a

it returns the errors message at n=3, of which a='electrocardiograms ' and b='electrocardiograms'.
Notably, there is an additional space in a, implying that the substring defined by start_index to end_index is consistently longer than the expected text. This discrepancy suggests an inconsistency in the indices relative to the data. Could you address this issue?