Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: how does data pushed in? #28

Open
brent-hoover opened this issue Jun 17, 2024 · 0 comments
Open

feat: how does data pushed in? #28

brent-hoover opened this issue Jun 17, 2024 · 0 comments
Labels
question Further information is requested

Comments

@brent-hoover
Copy link
Contributor

One of the main challenges you run into is that you need to get data into the app quickly. Doing it through api calls is good in many ways because:

  1. It's relatively standard
  2. Easy for partners to implement.

The only downside is that moving a LOT of data over HTTP is not always reliable or performant. Other ways like writing to a Kafka topic are quite a but more specific and also run into validation. Possibly some mix like a bulk api that just writes to a kafka topic that can take a lot of data at once. Think getting an inventory update every day for 25k products.

Other ways of course is uploading files like CSV or JSON or JSONL.

@brent-hoover brent-hoover added the question Further information is requested label Jun 17, 2024
@brent-hoover brent-hoover changed the title feat: how does data flow in? feat: how does data get pulled in? Jun 17, 2024
@brent-hoover brent-hoover changed the title feat: how does data get pulled in? feat: how does data pushed in? Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant