Skip to content

Latest commit

 

History

History
181 lines (151 loc) · 9.48 KB

README.md

File metadata and controls

181 lines (151 loc) · 9.48 KB

Rewatch

This directory contains ReScript's build system and CLI (rescript build, watch, clean, format, and compiler-args). It ships as rescript in the @rescript/<platform> packages and is tested with the integration suite in ../rewatch/tests.

The compiler is linked in: parses and compiles run in-process on a pool of OCaml domains rather than as bsc subprocesses (see In-process compilation).

Building and running

From the repository root, with the OPAM dependencies of rescript.opam installed:

make                                     # builds bsc and packages/@rescript/<platform>/bin/rescript.exe
dune build rewatch-ocaml/rescript_ocaml.exe   # just this executable

The npm launcher (cli/rescript.js) supplies the runtime path. To run the dune executable directly, point it at the runtime:

export RESCRIPT_RUNTIME="$PWD/packages/@rescript/runtime"
_build/default/rewatch-ocaml/rescript_ocaml.exe build path/to/project

Builds and rescript format use the embedded compiler. The bsc.exe next to rescript.exe is still recorded in lib/rescript/compiler-info.json, where editor tooling looks for the platform binaries.

Linux release builds use dune's static profile, so the published executable does not depend on the runner's libc. The version lives in rewatch_version.ml and is kept in sync with the compiler by yarn constraints --fix.

Layout

Area Modules
CLI and entry point cli, rescript_ocaml, output
Configuration config_types, config_decode (strict JSON decoding), config
Projects and packages project_context (workspace roots and dependency lookup), package_resolution, package_traversal, package_graph, package_plan, package_diagnostics
Sources source, source_filter (--filter), source_dirs
Build orchestration build (one build or watch rebuild), build_preparation, build_session (state kept across watch rebuilds), build_attempt (state of one attempt), build_state, build_report
Parsing and compiling package_parse, package_compilation, compiler_args, compiler_process (logical compiler jobs and their in-process execution), compiler_execution_mode (worker count), compiler_scheduler, compiler_info (fingerprints that decide when a package is cleaned), compile_assets, module_graph, graph
Embedded compiler compiler/bsc/rescript_compiler_driver (one request: argv in, exit code and captured output out), compiler/ext/compiler_request_state and compiler_request_output (per-request working directory and output capture)
Artifacts build_artifacts (output paths, publication, stale cleanup), build_freshness, clean, file_util
Processes process (worker pool), process_child (one child and its pipes), after_build
Platform platform.mli implemented by platform_unix.ml or platform_windows.ml (selected by dune), windows_job_stubs.c, termination_signals, build_lock
Watching watcher (rebuild loop), watch_scope (what to watch), watch_snapshot (filesystem snapshots), native_watcher (libuv notifications)
Other commands format, compiler_args_command

Native filesystem notifications only wake the watcher. The snapshot taken afterwards decides what changed, so merged, reordered, or dropped events can't cause missed rebuilds.

Artifacts

Each package keeps its artifacts flat in lib/rescript, keyed by the compiler's file name of each module (X or X-Ns): ASTs, .cmi, .cmj, .cmt and .cmti files, namespace maps, .compiler.log, compiler-info.json, .sourcedirs.json and, in the project that takes them, the build and watch locks. The compiler runs in that directory and writes there directly; dependencies are on its include path as their lib/rescript.

  • Freshness is timestamp-based: an AST newer than its source is current, a .cmt records the last compile, and a .cmi the last change of an interface.
  • ASTs, interfaces and typed trees are held in memory until their request finished (Build_file_store.set_held_writes), so a failed request leaves the previous AST and interface on disk. An interface is written only when it changed, which keeps the modification time that dependents compare with, and before the typed tree that marks the module compiled, so a build killed in between compiles the module again. A failed compile publishes its partial typed tree for editor tooling after dating its AST back.
  • Published files are renamed into place, so neither a reader nor a killed build sees part of one. On Windows the rename fails while an editor has the file open without delete sharing; the file is then written in place, and a failed write removes its partial file.
  • To make the next build parse a source again, the build dates its AST back instead of removing it: a later compile of the same build or watch session may still read it.
  • Stale public JavaScript is removed only when an AST shows that the build generated it for a source that is gone.
  • lib/bs and lib/ocaml, left by ReScript 12 and earlier, make the first build clean the package and are removed with it.

In-process compilation

Independent parse and compile requests run on a bounded pool of OCaml domains. The pool has min(8, max(1, CPUs - 1)) workers; set REWATCH_COMPILER_DOMAINS to choose another count: a positive integer, lowered to the machine's limit of max(12, min(32, CPUs)). The main domain schedules dependencies; workers run compiler requests and publish what they produce.

The runtime's allocation report at exit (OCAMLRUNPARAM=v=0x400) only counts the main domain. With REWATCH_REPORT_WORKER_ALLOCATIONS set, the build also prints worker_allocated_words: <n>, the words its workers allocated, which the compiler benchmarks (scripts/compilerBenchmarks.js) add to it.

The driver gives each request fresh compiler state (flags, warnings, caches, predefined types, identifier counters) and restores it on every exit, so requests on different domains don't interfere and repeated requests produce byte-identical artifacts. Compiler output is captured per request instead of going to the process's stdout and stderr. The compiler build identity is derived from the compiler sources, so compiler changes clean the build while rewatch-only changes don't.

External PPXs, js-post-build hooks, and --after-build remain subprocesses under the process owners described below.

Behavior notes

  • rescript.json maps that the build system interprets reject duplicate keys. A namespace-entry without a namespace is an error.
  • Configuration errors use the compiler-style form path: message, and JSON syntax errors use path:line:column: message. Messages name the field and the accepted values.
  • The CLI uses Cmdliner: help is rendered as a man page, --no-timing takes no value, help takes precedence over version, and --version also works after a subcommand.
  • --filter takes a Perl-style regular expression as the Re library reads it. Constructs it cannot match, such as backreferences, are rejected with an error. Non-ASCII names are matched byte-wise.
  • A failing --after-build command fails the build. On Unix, hooks inherit redirected stdin but not an interactive terminal, because they run in their own process group.
  • format -v and compiler-args -v write their logs to stderr, so stdout remains formatted source or JSON.
  • While the root configuration is invalid, only configuration changes trigger recovery attempts in watch mode.
  • Watch mode notices a missing dependency being installed into an existing node_modules of an ancestor directory, where the dependency lookup finds it.
  • A damaged published AST, such as one cut short by a killed build, makes its module parse again.

Platforms

  • Unix: children are started with spawn in their own process groups. SIGINT, SIGTERM, SIGHUP, and SIGQUIT terminate those groups before rescript exits.
  • Windows: children are started through windows_job_stubs.c, which assigns each child to a Job Object before it runs. Cancellation then also ends its descendants.
  • Pipes: each child's stdout and stderr are drained by reader threads, because Windows can't select on anonymous pipes.
  • Watching: uses libuv through luv. rescript.opam.template pins luv to a commit of the rescript-lang/luv fork (libuv 1.53).

The test suite compiles platform_windows.ml on every platform, so the Windows backend is always type-checked. Its process and Job Object paths only run on Windows CI.

Testing

dune runtest tests/rewatch_ounit_tests                          # unit tests
sh rewatch-ocaml/tests/run.sh "$PWD/packages/@rescript/<platform>/bin/rescript.exe"   # focused end-to-end tests
make test-rewatch                                               # integration suite in rewatch/tests

The scripts in tests/check_*.sh check configuration acceptance, command validation, and verbose and interactive output, partly against snapshots in tests/snapshots/. Run them with UPDATE_SNAPSHOTS=1 to rewrite the snapshots after an intended output change. tests/check_canonical_test_coverage.sh ensures that every integration test script is run by rewatch/tests/suite.sh.

Packaging

Each platform package contains bin/rescript.exe, together with THIRD_PARTY_NOTICES_REWATCH.md and RE_LICENSE.md for the bundled OCaml libraries. After a build, node scripts/updateArtifactList.js must leave packages/artifacts.json unchanged.