Btukfyl

I am currently trying to read a bson file to import it into a database. I can already read the file and print it as as bytes, but I end up just getting an bson.errors.InvalidBSON: objsize too large error.

This is the code that is trying to decode the file

with zip.open(name) as myfile:

    content = myfile.read()

    print(content)

    print(bson.decode_all(content))

and this is the output i get

b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'

Traceback (most recent call last):

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/bin/mongo-backup", line 11, in <module>

    load_entry_point('mongo-backup-cli', 'console_scripts', 'mongo-backup')()

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 764, in __call__

    return self.main(*args, **kwargs)

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 717, in main

    rv = self.invoke(ctx)

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 1137, in invoke

    return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 956, in invoke

    return ctx.invoke(self.callback, **ctx.params)

  File "/home/jonas/.envs/mongodb-backup-py-aGZYxULQ/lib/python3.6/site-packages/click/core.py", line 555, in invoke

    return callback(*args, **kwargs)

  File "/home/jonas/work/mongodb-backup-py/mongo_backup/cli.py", line 45, in restore

    restore_files(uri, db_name, file.name)

  File "/home/jonas/work/mongodb-backup-py/mongo_backup/restore.py", line 6, in restore_files

    print(read_zip_file(file))

  File "/home/jonas/work/mongodb-backup-py/mongo_backup/zip.py", line 24, in read_zip_file

    print(bson.decode_all(content))

bson.errors.InvalidBSON: objsize too large

print(content)



b'[{"_id": {"$oid": "5bf3cf511c9d44000088c376"}, "some": "sort of"}, {"_id": {"$oid": "5bf3cf5c1c9d44000088c377"}, "test": "data"}]'

the bytes in the content variable is json encoded bson, not plain bson

if this is the output format you intend to continue to use, you need to change your code to use bson's JSON utility to load the string into a python object:

with zip.open(name) as myfile:

    content = myfile.read()

    print(content)

    print(bson.json_util.loads(content))

    #         ^--------------^

    #                | this stuff