numpy arrays ...

[lal.george@…]

What is the best way to transfer very large numpy arrays using PyYAML?

Ideally I would like to do something like:

m = ; # a very large numpy array

y = "---\nx : '%s'\n" % (m.dumps())

z = yaml.load(y)

However, the yaml.load returns an exception:

'utf8 codec can't decode byte #x80: unexpected code byte in "<string>", position 9.

Since I have quoted the string, yaml should not be diving into it - correct?

Thanks.

Best,

Lal

[xi]

YAML specification requires that the input stream (or in this case, the input string) is encoded in utf-8 or utf-16. That's why the parser complains when it doesn't recognize a valid utf8 character.

In your case, you'd better use pyyaml to serialize the value:

y = yaml.dump({'x': m.dumps()}) # you could add the parameter default_flow_style=False

Alternatively, you could write

y = "---\nx: !!binary '%s'" % m.dumps().encode('base64')