Ticket #73 (closed defect: invalid)

Opened 6 years ago

Last modified 6 years ago

numpy arrays ...

Reported by:… Owned by: xi
Priority: high Component: pyyaml
Severity: normal Keywords:


What is the best way to transfer very large numpy arrays using PyYAML?

Ideally I would like to do something like:

m = ; # a very large numpy array

y = "---\nx : '%s'\n" % (m.dumps())

z = yaml.load(y)

However, the yaml.load returns an exception:

'utf8 codec can't decode byte #x80: unexpected code byte in "<string>", position 9.

Since I have quoted the string, yaml should not be diving into it - correct?





Change History

comment:1 Changed 6 years ago by xi

  • Status changed from new to closed
  • Resolution set to invalid

YAML specification requires that the input stream (or in this case, the input string) is encoded in utf-8 or utf-16. That's why the parser complains when it doesn't recognize a valid utf8 character.

In your case, you'd better use pyyaml to serialize the value:

y = yaml.dump({'x': m.dumps()}) # you could add the parameter default_flow_style=False

Alternatively, you could write

y = "---\nx: !!binary '%s'" % m.dumps().encode('base64')

Add a comment

Modify Ticket

Change Properties
<Author field>
as closed
The resolution will be deleted. Next status will be 'reopened'

E-mail address and user name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.