Idl parser#90
Merged
Merged
Conversation
Pull Request Test Coverage Report for Build 599
💛 - Coveralls |
seriyps
commented
Jul 23, 2019
seriyps
force-pushed
the
idl-parser
branch
2 times, most recently
from
March 14, 2020 22:49
88a4cf8 to
2b2406b
Compare
Seems the only things left are some of annotations and docstrings
Now it's possible to use basic .avdl to encode/decode avro
Lexer fixes:
- Support empty string literals ("") — previously caused illegal token error
- Support backslash escape sequences in strings (\", \\, \n, \t, \r)
Parser fixes:
- Allow annotations and multiple doc comments before function/message
definitions, consistent with how enum/record/error/fixed declarations
are handled (meta function rule instead of doc_v function)
- Allow empty record and error type bodies (record Foo {})
- The meta function fix also resolved the undeclared 3rd shift/reduce
conflict, so Expect 2 is now correct
- process_imports/2 now loads all three import types instead of silently
dropping them; imported types are returned as {avsc, Map} tagged tuples
so they pass through protocol_to_avpr/2 without re-conversion
- load_idl_import/2 uses the imported file's dirname as cwd so nested
relative imports resolve correctly
- Add tests for each import type and for nested directory imports
Detect ".avdl" extension and delegate to avro_idl:decode_schema/2, then flatten the resulting union type and add each named type to the store individually via do_add_type/2.
Add a {read_fun, fun((Cwd, Path) -> {ok, Bin} | {error, term()})}
option to str_to_avpr/3 and decode_schema/3. The default read_fun
uses file:read_file/1. The fun is threaded through recursive IDL
imports so all file I/O can be intercepted at every level.
Also add decode_schema/3 as a convenience wrapper accepting the
same options.
- Add encode_decode_test/0 in avro_idl_tests covering: - types from imported .avdl (idl import) - types from imported .avpr (protocol import) - types from imported .avsc (schema import) - cross-namespace type references (org.erlang.ftp.MyAnnotated) - Fix protocol_with_typedefs.avdl: MyAnnotated referenced MyError (an error type filtered out by decode_schema), replaced with a valid reference to org.erlang.www.MyEnum2
Member
Author
|
@zmstone this is now ready. It took me some time to finalize it 😄 |
mikpe
reviewed
Mar 6, 2026
mikpe
approved these changes
Mar 6, 2026
mikpe
left a comment
Member
There was a problem hiding this comment.
With the caveat wrt the record name.
The `record` would be a reserved word and would warn in OTP-29; the rest of the records are updated for consistency
zmstone
reviewed
Mar 6, 2026
zmstone
approved these changes
Mar 6, 2026
mikpe
approved these changes
Mar 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implementation of idl parser. Fixes #68.
Usage Examples
Load a schema file and encode/decode
Convert .avdl to an AVPR map (JSON protocol representation)
{ok, Bin} = file:read_file("my_service.avdl"), Avpr = avro_idl:str_to_avpr( binary_to_list(Bin), filename:dirname("my_service.avdl")), %% Avpr is a map: #{<<"protocol">> => ..., <<"types">> => [...], <<"messages">> => ...}In-memory schemas (useful for tests or embedded schemas)