Step-by-step setup of Webpack for a typical React app: JSX, TypeScript, JS and CSS in separate files with hashes in names, Babel transpilation for .browserslistrc, polyfills for older browsers, hot reloading during dev
Take a look at git history for the step-by-step setup
If I were developing a new React app, I would consider Vite, built on top of great esbuild