From f9051327fdfc3240993c2c851572706b38b457da Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Mon, 28 Jul 2025 23:04:24 +0000 Subject: [PATCH 1/4] docs: Update concepts to cover RFC 168 Uncomment * Converter type classes * VTable types * deepCopy --- doc/manual_experimental.md | 204 ++++++++++++++++++++++++++++++++++++- 1 file changed, 201 insertions(+), 3 deletions(-) diff --git a/doc/manual_experimental.md b/doc/manual_experimental.md index 81defd70b5304..0790f78389a62 100644 --- a/doc/manual_experimental.md +++ b/doc/manual_experimental.md @@ -980,6 +980,24 @@ to describe usage protocols that do not reveal implementation details. Much like generics, concepts are instantiated exactly once for each tested type and any static code included within the body is executed only once. +Concepts were overhauled with RFC #168 , +so that generic code can be type-checked at declaration time rather than only at instantiation. +The redesign focuses on making concepts easier to understand and implement, +while providing an escape hatch that allows adding debug or log statements in generic code +without requiring extensive type constraint modifications. +It ensures backward compatibility by allowing old code with unconstrained generic parameters to continue working. +The implementation avoids relying on system.compiles, which is under-specified, +tightly coupled to Nim’s current implementation, and slow. + +The redesign does not aim to support every detail of the old concept design, +relying instead on the escape hatch and underspecified types for flexibility. +It does not support accidental or 'hacky' features (e.g., specifying calling conventions within concepts), +nor does it support complex cross-parameter constraints like comparing sizes of parameters. +Such cases can be handled separately with `enableif` without complicating the core concept design. +Finally, the redesign does not turn concepts into interfaces, as this can already be done with macros; +instead, the declarative nature of the new concepts makes them easier to process with tooling and macros. + +The new style is covered below in subsections with a star (`*`) in the name. Concept diagnostics ------------------- @@ -1249,7 +1267,7 @@ object inheritance syntax involving the `of` keyword: # matching the BidirectionalGraph concept ``` -.. + Converter type classes ---------------------- @@ -1284,7 +1302,7 @@ object inheritance syntax involving the `of` keyword: ``` -.. + VTable types ------------ @@ -1342,7 +1360,7 @@ object inheritance syntax involving the `of` keyword: the `vtptr` magic produced types bound to `ptr` types. -.. + deepCopy -------- `=deepCopy` is a builtin that is invoked whenever data is passed to @@ -1362,6 +1380,186 @@ object inheritance syntax involving the `of` keyword: The builtin `deepCopy` can even clone closures and their environments. See the documentation of [spawn][spawn statement] for details. +Atoms and containers* +-------------------- +Concepts come in two forms: Atoms and containers. A container is a generic +concept like `Iterable[T]`, an atom always lacks any kind of generic parameter +(as in `Comparable`). + +Syntactically a concept consists of a list of proc and iterator declarations. +There are 3 syntatic additions: + +- `Self` is a builtin type within the concept's body stands for the current concept. +- `each` is used to introduce a generic parameter `T` within the concept's body + that is not listed within the concept's generic parameter list. +- `either orelse` is used to provide basic support for optional procs within a concept. + +We will see how these are used in the examples. + +Atoms* +----- + ```nim + type + Comparable = concept # no T, an atom + proc cmp(a, b: Self): int + + ToStringable = concept + proc `$`(a: Self): string + + Hashable = concept + proc hash(x: Self): int + proc `==`(x, y: Self): bool + + Swapable = concept + proc swap(x, y: var Self) + ``` +`Self` stands for the currently defined concept itself. It is used to avoid a +recursion, `proc cmp(a, b: Comparable): int` is invalid. + +Containers* +---------- +A container has at least one generic parameter (most often called `T`). +The first syntactic usage of the generic parameter specifies how to infer and +bind `T`. Other usages of `T` are then checked to match what it was bound to. + + ```nim + type + Indexable[T] = concept # has a T, a collection + proc `[]`(a: Self; index: int): T # we need to describe how to infer 'T' + # and then we can use the 'T' and it must match: + proc `[]=`(a: var Self; index: int; value: T) + proc len(a: Self): int + ``` +Nothing interesting happens when we use multiple generic parameters: + ```nim + type + Dictionary[K, V] = concept + proc `[]`(a: Self; key: K): V + proc `[]=`(a: var Self; key: K; value: V) + ``` +The usual `: Constraint` syntax can be used to add generic constraints to the +involved generic parameters: + ```nim + type + Dictionary[K: Hashable; V] = concept + proc `[]`(a: Self; key: K): V + proc `[]=`(a: var Self; key: K; value: V) + ``` + +each T* +------ +Note: `each T` is currently not implemented. + +`each T` allows to introduce generic parameters that are not part of a concept's +generic parameter list. It is furthermore a special case to allow for the +common "every field has to fulfill property P" scenario: + + ```nim + type + Serializable = concept + iterator fieldPairs(x: Self): (string, each T) + proc write(x: T) + + proc writeStuff[T: Serializable](x: T) = + for name, field in fieldPairs(x): + write name + write field + ``` + +either orelse* +------------- +Note: `either orelse` is currently not implemented. + +In generic code it's often desirable to specialize the code in an ad-hoc +manner. `system.addQuoted` is an example of this: + ```nim + proc addQuoted[T](dest: var string; x: T) = + when compiles(dest.add(x)): + dest.add(x) + else: + dest.add($x) + ``` +If we want to describe `T` with a concept we need some way to describe optional +aspects. `either orelse` can be used: + ```nim + type + Quatable = concept + either: + proc $(x: Self): string + orelse: + proc add(s: var string; elem: self) + + proc addQuoted[T: Quotable](s: var string; x: T) = + when compiles(s.add(x)): + s.add(x) + else: + s.add($x) + ``` + +More examples +------------- +**system.find** + + ```nim + type + Findable[T] = concept + iterator items(x: Self): T + proc `==`(a, b: T): bool + + proc find[T](x: Findable[T]; elem: T): int = + var i = 0 + for a in x: + if a == elem: return i + inc i + result = -1 + ``` + +**Sortable** + +Note that a declaration like + ```nim + type + Sortable[T] = Indexable[T] and T is Comparable and T is Swapable + ``` +is possible but unwise. The reason is that `Indexable` either contains too many +procs we don't need or accessors that are slightly off as they don't offer the +right kind of mutability access. Here is the proper definition: + ```nim + type + Sortable[T] = concept + proc `[]`(a: var Self; b: int): var T + proc len(a: Self): int + proc swap(x, y: var T) + proc cmp(a, b: T): int + ``` + +Concept matching* +---------------- +A type `T` matches a concept `C` if every proc and iterator header `H` of `C` +matches an entity `E` in the current scope. + +The matching process is forgiving: + +- If `H` is a proc, `E` can be a proc, a func, a method, a template, a converter or a macro. +- `E` can have more parameters than `H` as long as these parameters have default values. +- The parameter names do not have to match. +- If `H` has the form `proc p(x: Self): T` then `E` can be a public object field of name `p` and of type `T`. +- If `H` is an iterator, `E` must be an iterator too, but `E`'s parameter names do not have to match and it can have additional default parameters. + +Escape hatch* +------------ +Generic routines that have at least one concept parameter are type-checked at +declaration time. To disable type-checking in certain code sections an +`untyped` block can be used: + ```nim + proc sort(x: var Sortable) = + ... + # damn this sort doesn't work, let's find out why: + untyped: + # no need to change 'Sortable' so that it mentions '$' for the involved + # element type! + echo x[i], " ", x[j] + ``` Dynamic arguments for bindSym ============================= From 236d42e66f52a21a89438eaceb11585bdf6d4a46 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Emre=20=C5=9Eafak?= Date: Mon, 28 Jul 2025 19:32:37 -0400 Subject: [PATCH 2/4] linting --- doc/manual_experimental.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/manual_experimental.md b/doc/manual_experimental.md index 0790f78389a62..0ca274f1f48b2 100644 --- a/doc/manual_experimental.md +++ b/doc/manual_experimental.md @@ -1381,7 +1381,7 @@ object inheritance syntax involving the `of` keyword: the documentation of [spawn][spawn statement] for details. Atoms and containers* --------------------- +--------------------- Concepts come in two forms: Atoms and containers. A container is a generic concept like `Iterable[T]`, an atom always lacks any kind of generic parameter (as in `Comparable`). @@ -1447,7 +1447,7 @@ involved generic parameters: ``` each T* ------- +------- Note: `each T` is currently not implemented. `each T` allows to introduce generic parameters that are not part of a concept's @@ -1467,7 +1467,7 @@ common "every field has to fulfill property P" scenario: ``` either orelse* -------------- +-------------- Note: `either orelse` is currently not implemented. In generic code it's often desirable to specialize the code in an ad-hoc @@ -1496,8 +1496,8 @@ aspects. `either orelse` can be used: s.add($x) ``` -More examples -------------- +More examples* +-------------- **system.find** ```nim @@ -1534,7 +1534,7 @@ right kind of mutability access. Here is the proper definition: ``` Concept matching* ----------------- +----------------- A type `T` matches a concept `C` if every proc and iterator header `H` of `C` matches an entity `E` in the current scope. @@ -1547,7 +1547,7 @@ The matching process is forgiving: - If `H` is an iterator, `E` must be an iterator too, but `E`'s parameter names do not have to match and it can have additional default parameters. Escape hatch* ------------- +------------- Generic routines that have at least one concept parameter are type-checked at declaration time. To disable type-checking in certain code sections an `untyped` block can be used: From 8a27540d671731c12920f74b2458673c1372f930 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Emre=20=C5=9Eafak?= Date: Mon, 28 Jul 2025 19:37:31 -0400 Subject: [PATCH 3/4] linting --- doc/manual_experimental.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/manual_experimental.md b/doc/manual_experimental.md index 0ca274f1f48b2..6561dad4e7c7d 100644 --- a/doc/manual_experimental.md +++ b/doc/manual_experimental.md @@ -1397,7 +1397,7 @@ There are 3 syntatic additions: We will see how these are used in the examples. Atoms* ------ +------ ```nim type Comparable = concept # no T, an atom @@ -1417,7 +1417,7 @@ Atoms* recursion, `proc cmp(a, b: Comparable): int` is invalid. Containers* ----------- +----------- A container has at least one generic parameter (most often called `T`). The first syntactic usage of the generic parameter specifies how to infer and bind `T`. Other usages of `T` are then checked to match what it was bound to. From 811e3947d01463e84f92844406b642779b21d348 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Emre=20=C5=9Eafak?= Date: Tue, 29 Jul 2025 00:56:51 -0400 Subject: [PATCH 4/4] Remove content in response to comments by @araq --- doc/manual_experimental.md | 185 +------------------------------------ 1 file changed, 2 insertions(+), 183 deletions(-) diff --git a/doc/manual_experimental.md b/doc/manual_experimental.md index 6561dad4e7c7d..f030f93884d1f 100644 --- a/doc/manual_experimental.md +++ b/doc/manual_experimental.md @@ -982,15 +982,12 @@ and any static code included within the body is executed only once. Concepts were overhauled with RFC #168 , so that generic code can be type-checked at declaration time rather than only at instantiation. -The redesign focuses on making concepts easier to understand and implement, -while providing an escape hatch that allows adding debug or log statements in generic code -without requiring extensive type constraint modifications. +The redesign focuses on making concepts easier to understand and implement. It ensures backward compatibility by allowing old code with unconstrained generic parameters to continue working. The implementation avoids relying on system.compiles, which is under-specified, tightly coupled to Nim’s current implementation, and slow. -The redesign does not aim to support every detail of the old concept design, -relying instead on the escape hatch and underspecified types for flexibility. +The redesign does not aim to support every detail of the old concept design. It does not support accidental or 'hacky' features (e.g., specifying calling conventions within concepts), nor does it support complex cross-parameter constraints like comparing sizes of parameters. Such cases can be handled separately with `enableif` without complicating the core concept design. @@ -1267,119 +1264,6 @@ object inheritance syntax involving the `of` keyword: # matching the BidirectionalGraph concept ``` - - Converter type classes - ---------------------- - - Concepts can also be used to convert a whole range of types to a single type or - a small set of simpler types. This is achieved with a `return` statement within - the concept body: - - ```nim - type - Stringable = concept x - $x is string - return $x - - StringRefValue[CharType] = object - base: ptr CharType - len: int - - StringRef = concept x - # the following would be an overloaded proc for cstring, string, seq and - # other user-defined types, returning either a StringRefValue[char] or - # StringRefValue[wchar] - return makeStringRefValue(x) - - # the varargs param will here be converted to an array of StringRefValues - # the proc will have only two instantiations for the two character types - proc log(format: static string, varargs[StringRef]) - - # this proc will allow char and wchar values to be mixed in - # the same call at the cost of additional instantiations - # the varargs param will be converted to a tuple - proc log(format: static string, varargs[distinct StringRef]) - ``` - - - - VTable types - ------------ - - Concepts allow Nim to define a great number of algorithms, using only - static polymorphism and without erasing any type information or sacrificing - any execution speed. But when polymorphic collections of objects are required, - the user must use one of the provided type erasure techniques - either common - base types or VTable types. - - VTable types are represented as "fat pointers" storing a reference to an - object together with a reference to a table of procs implementing a set of - required operations (the so called vtable). - - In contrast to other programming languages, the vtable in Nim is stored - externally to the object, allowing you to create multiple different vtable - views for the same object. Thus, the polymorphism in Nim is unbounded - - any type can implement an unlimited number of protocols or interfaces not - originally envisioned by the type's author. - - Any concept type can be turned into a VTable type by using the `vtref` - or the `vtptr` compiler magics. Under the hood, these magics generate - a converter type class, which converts the regular instances of the matching - types to the corresponding VTable type. - - ```nim - type - IntEnumerable = vtref Enumerable[int] - - MyObject = object - enumerables: seq[IntEnumerable] - streams: seq[OutputStream.vtref] - - proc addEnumerable(o: var MyObject, e: IntEnumerable) = - o.enumerables.add e - - proc addStream(o: var MyObject, e: OutputStream.vtref) = - o.streams.add e - ``` - - The procs that will be included in the vtable are derived from the concept - body and include all proc calls for which all param types were specified as - concrete types. All such calls should include exactly one param of the type - matched against the concept (not necessarily in the first position), which - will be considered the value bound to the vtable. - - Overloads will be created for all captured procs, accepting the vtable type - in the position of the captured underlying object. - - Under these rules, it's possible to obtain a vtable type for a concept with - unbound type parameters or one instantiated with metatypes (type classes), - but it will include a smaller number of captured procs. A completely empty - vtable will be reported as an error. - - The `vtref` magic produces types which can be bound to `ref` types and - the `vtptr` magic produced types bound to `ptr` types. - - - - deepCopy - -------- - `=deepCopy` is a builtin that is invoked whenever data is passed to - a `spawn`'ed proc to ensure memory safety. The programmer can override its - behaviour for a specific `ref` or `ptr` type `T`. (Later versions of the - language may weaken this restriction.) - - The signature has to be: - - ```nim - proc `=deepCopy`(x: T): T - ``` - - This mechanism will be used by most data structures that support shared memory, - like channels, to implement thread safe automatic memory management. - - The builtin `deepCopy` can even clone closures and their environments. See - the documentation of [spawn][spawn statement] for details. - Atoms and containers* --------------------- Concepts come in two forms: Atoms and containers. A container is a generic @@ -1446,56 +1330,6 @@ involved generic parameters: proc `[]=`(a: var Self; key: K; value: V) ``` -each T* -------- -Note: `each T` is currently not implemented. - -`each T` allows to introduce generic parameters that are not part of a concept's -generic parameter list. It is furthermore a special case to allow for the -common "every field has to fulfill property P" scenario: - - ```nim - type - Serializable = concept - iterator fieldPairs(x: Self): (string, each T) - proc write(x: T) - - proc writeStuff[T: Serializable](x: T) = - for name, field in fieldPairs(x): - write name - write field - ``` - -either orelse* --------------- -Note: `either orelse` is currently not implemented. - -In generic code it's often desirable to specialize the code in an ad-hoc -manner. `system.addQuoted` is an example of this: - ```nim - proc addQuoted[T](dest: var string; x: T) = - when compiles(dest.add(x)): - dest.add(x) - else: - dest.add($x) - ``` -If we want to describe `T` with a concept we need some way to describe optional -aspects. `either orelse` can be used: - ```nim - type - Quatable = concept - either: - proc $(x: Self): string - orelse: - proc add(s: var string; elem: self) - - proc addQuoted[T: Quotable](s: var string; x: T) = - when compiles(s.add(x)): - s.add(x) - else: - s.add($x) - ``` - More examples* -------------- **system.find** @@ -1546,21 +1380,6 @@ The matching process is forgiving: - If `H` has the form `proc p(x: Self): T` then `E` can be a public object field of name `p` and of type `T`. - If `H` is an iterator, `E` must be an iterator too, but `E`'s parameter names do not have to match and it can have additional default parameters. -Escape hatch* -------------- -Generic routines that have at least one concept parameter are type-checked at -declaration time. To disable type-checking in certain code sections an -`untyped` block can be used: - ```nim - proc sort(x: var Sortable) = - ... - # damn this sort doesn't work, let's find out why: - untyped: - # no need to change 'Sortable' so that it mentions '$' for the involved - # element type! - echo x[i], " ", x[j] - ``` - Dynamic arguments for bindSym =============================