Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chart rendering breaks around 5000 x-Axis data points #523

Open
bvogel opened this issue May 14, 2024 · 21 comments
Open

chart rendering breaks around 5000 x-Axis data points #523

bvogel opened this issue May 14, 2024 · 21 comments

Comments

@bvogel
Copy link

bvogel commented May 14, 2024

node v20
highcharts version 10.4.0
export server version: 3.1.1

Expected behaviour

Graph rendered correctly (4999 data points)

image

Actual behaviour

Graph rendered with increasing errors (5000 data points)

image

graph don't start at the same point any more

Graph broken (5001 data points)

image

graph is missing 3/4 of data

Reproduction steps

infiles for the cases:

working json

partially broken

broken

@bvogel
Copy link
Author

bvogel commented May 14, 2024

In the meantime I found out that the error in the second graph may result from unbalanced series lengths, so that error may be expected, however the series have the same length for the third graph and it's still broken

@bvogel
Copy link
Author

bvogel commented May 14, 2024

I can add an additional example for failing data around the 5000 data points mark:

working (4999 data points)
image

broken (5000 data points)
image

Although the graph is still being rendered it looses the requested gradient.

@jszuminski
Copy link
Contributor

Thanks for reporting @bvogel!

It seems to be a duplicate of this issue: #490

It should work in the newest version.

@bvogel
Copy link
Author

bvogel commented May 16, 2024

Thank you, any news on the release date?

I'd be happy to test a beta or rc release as well

@bvogel
Copy link
Author

bvogel commented Jun 20, 2024

Another month has passed, I can't really take this issue as resolved as we can't use the proposed solution - it still hasn't been released yet. Please advise how to proceed. (Due to internal guidelines we aren't in a position to use branches as reference for our packages)

@bvogel
Copy link
Author

bvogel commented Jun 25, 2024

bump!

@jszuminski
Copy link
Contributor

@bvogel it has been fixed (on the master branch) and on the official deployed Highcharts Export Server. Please try it out :)

@bvogel
Copy link
Author

bvogel commented Jul 10, 2024

@jszuminski unfortunately we can't use the external server due to company policies, so I'll have to wait for the official release (my test data was deleted already)

@jszuminski
Copy link
Contributor

Absolutely, I understand. If you'd like, you can always try to download the package from GitHub.

@bvogel
Copy link
Author

bvogel commented Aug 9, 2024

Hi, I just tested with:

INFO 2024-08-08T18:05:17.669282664Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [verbose] - [cache] Fetching script - https://code.highcharts.com/11.4/highcharts-more.js
INFO 2024-08-08T18:05:17.608273297Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [verbose] - [cache] Fetching script - https://code.highcharts.com/11.4/highcharts.js
INFO 2024-08-08T18:05:17.602820134Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [cache] Updating cache version to Highcharts: 11.4/.
INFO 2024-08-08T18:05:17.602413336Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [cache] Fetching and caching Highcharts dependencies.
INFO 2024-08-08T18:05:17.600292924Z [resource.labels.containerName: highcharts-export-server] Thu Aug 08 2024 18:05:17 GMT+0000 [notice] - [process] Attaching exit listeners to the process.
INFO 2024-08-08T18:05:17.598449415Z [resource.labels.containerName: highcharts-export-server] Starting Highcharts Export Server v4.0.1...

but still seeing the same error. 5000 x-axis points will break rendering.

In contrast to #490 I don't get an 500 error but just a broken graph.

@bvogel
Copy link
Author

bvogel commented Aug 11, 2024

i.e. can we have this re-opened please?

@jszuminski jszuminski reopened this Aug 15, 2024
@jszuminski
Copy link
Contributor

I've reopened this issue and added it to our backlog. If the problem indeed persists (we will have to investigate that), we will try to make it a part of one of the upcoming releases.

@bvogel
Copy link
Author

bvogel commented Aug 15, 2024

Thank you!

@bvogel
Copy link
Author

bvogel commented Aug 23, 2024

@jszuminski were you able to reproduce? should I sent test data from our end?

@jszuminski
Copy link
Contributor

@bvogel if possible, you can share the test data, that would be helpful. You can also try to create a minimal reproducible example in JSFiddle, starting from there: https://jsfiddle.net/ftpyxewh/

@bvogel
Copy link
Author

bvogel commented Sep 2, 2024

Hi @jszuminski,

this fiddle will work as fiddle but is broken if exported through the server https://jsfiddle.net/5ejsu9go/

this one will work: https://jsfiddle.net/j6ns0d3v/

Some additional data: the export server receives a post with:

json: { infile: <options from the fiddle>, scale: 2, type: "jpeg" }

@bvogel
Copy link
Author

bvogel commented Sep 12, 2024

@jszuminski any news on this, were you able to reproduce the error we are seeing?

@jszuminski
Copy link
Contributor

@bvogel I seem to have different errors when exporting this, but do not encounter what you encounter. The good news is that this #561 pull request will introduce many beneficial changes and will allow to detect errors in request/CLI options if they are the ones causing an error so a part of the problem will get solved there; you will get an error or a warning message if the configuration is not optimal or not supported.

However, for now, let's get back to the main problem which is the partial rendering of the line chart. Could you please tell me:

  1. Are you sending an HTTP request or are you doing this via CLI? currently, There are separate exporting processes happening; we're currently working on unifying them which will also get rid of this problem.

  2. Could you share the full CLI command and a broken .json file so that I can 100% get the error that you get? I will then just run the command and share the result with you to compare. If this causes an error on my side as well, I will debug it and get to the root cause.

@bvogel
Copy link
Author

bvogel commented Sep 13, 2024

we use a HTTP POST request, so I have no CLI or json file. The jsfiddles are very close to the requests we send. as mentioned in the previous post we use this code:

    def request_chart(for_pdf: false)
      scaling = for_pdf ? 2 : 1
      result = HTTP.post(HIGHCHARTS_EXPORT_SERVER_URL, json: { infile: @chart_options, scale: scaling, type: "jpeg" })

      raise HighchartsError, "status: #{result.status.code} reason: #{result.body}" unless result.status.success?

      StringIO.new(result.body)
    end

^^^ we see the error for for_pdf = true i.e. scale: 2

with the @chart_options being 100% identical to the ones in the fiddles. We use Highcharts Export Server v4.0.1 with Highcharts library 11.4.0

edit: this is the log from a export server for a failing request

2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Got an incoming HTTP request with ID d510f5bd30cc4ad98ad04fd4ba82ae58.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [chart] Starting the exporting process.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [chart] Attempting to export from a raw input.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Work received, starting to process.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Acquiring a worker handle.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Acquired a worker handle.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Starting work on pool entry with ID c7d29e24-f160-4ac3-befe-28c17e54e271.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Determining export path.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [export] Treating as config.
2024-09-13 14:42:26 Fri Sep 13 2024 12:42:26 GMT+0000 [verbose] - [pool] Work completed in 129 ms.

edit 2: you can use this link to get a broken sample: https://capinside.com/products/pictet-digital-hi-eur-lu0386392772/fact_sheet?performance_start_date=2011-01-05&performance_end_date=2024-09-13&peergroup_id=13&add_personalization=false&risk_return_range=1_year and this one (one day less) for a working one https://capinside.com/products/pictet-digital-hi-eur-lu0386392772/fact_sheet?performance_start_date=2011-01-06&performance_end_date=2024-09-13&peergroup_id=13&add_personalization=false&risk_return_range=1_year - and for good measure, this is the page we start from, probably some data can be deduced from the graph in use there: https://capinside.com/p/pictet-digital-hi-eur-lu0386392772?locale=en

@jszuminski
Copy link
Contributor

Okay, thank you very much @bvogel! I have successfully managed to reproduce it locally and get the same result as you do:

Screenshot 2024-09-18 at 09 06 48

I will now investigate the cause of the problem by debugging what's going on inside the Puppeteer.

@bvogel
Copy link
Author

bvogel commented Sep 18, 2024

Thank you so much for the update!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants