Repository navigation
How to read ndjson file in assets folder #1
Description
Activity
If you are loading the ndjson at once you can just:
final ndjsonSource = Stream.fromIterable(<String>[await rootBundle.loadString("assets/data/object.ndjson")]); final Stream<NdjsonLine> ndjson = ndjsonSource.parseNdjson();
Keep in mind it is loading all the ndjson data at once, which kills the purpose of the ndjson which is to lazy-load JSON entries. But if you just want to make it run, the above should work.
I resolved this by:
- reading the assets file as ByteData with rootbundle.load(), writing the file to ApplicationDocumentsDirectory (if not already done).
- the ndjson file with Stream<List> ndjsonSource = File().open() and then parsed with .parseNdjsonWithConverter().
If you are loading the ndjson at once you can just:
final ndjsonSource = Stream.fromIterable([await rootBundle.loadString("assets/data/object.ndjson")]);
final Stream ndjson = ndjsonSource.parseNdjson();
Keep in mind it is loading all the ndjson data at once, which kills the purpose of the ndjson which is to lazy-load JSON entries. But if you just want to make it run, the above should work.Thanks @alexcmgit I'll try this also and compare the load speed. I call this function in initState() as the app depends on the ndjson file loading.
I run .parseNdjsonWithConverter() within compute() to isolate the process given the file size. However, as isolate() is unable to take a stream argument or access assets, I'll continue with the one-time write of the asset to appDocDir, that way I can run the .parseNdjsonWithConverter() within compute(). Seems like an inefficient hack, but it works.
I would like to use ndjson package to read a large ndjson file stored in the assets folder of my flutter app.
As a simple example I have copied your object.ndjson to assets/data folder. Here are relevant code parts:
pubspec.yaml:
main.dart:
This has the issue that "The method 'parseNdjson' isn't defined for the type 'String'..." because parseNdjson expects a stream. How can I use your ndjson flutter package to read a file in the assets folder?