really slow on large input ...

[lal.george@…]

yaml.load appears to be really (really) slow on large input. My string is

--- seqno : 1 outof : 1 result :

• <1.6Mbyte string>

Both the input and output are fairly simple. Why does it take a long time to parse?

Lal

[anonymous]

is this with libyaml?

[xi]

Have you tried to parse the file with a libyaml based parser?

import yaml
yaml.load(input, Loader=yaml.CLoader)

[anonymous]

Indeed, someone kindly suggested this. However, I am having trouble installing with libyaml.

I managed to build LibYaml? and Pyrex successfully, however when I try and install yaml with libyaml using:

python setup_with_libyaml.py install

I get the errors shown below.

It's complaining about no yaml.h file which is correct, and also a syntax error at the lines:

71 struct pyx_obj_5_yaml_CParser { 72 PyObject_HEAD 73 struct pyx_vtabstruct_5_yaml_CParser *pyx_vtab; 74 yaml_parser_t parser;

Any thoughts. My version of gcc is 3.2.3!

Any help is appreciated.

Best,

Lal

[xi]

Could you attach the pyrex-generated C source and post the complete output of the compiler?

[anonymous]

xi,

I finally managed to build with LibYaml?, but it was a real pain ... The primary issues had to do with the options to gcc for the include path to yaml.h, and the library path to libyaml.so.

I haven't include the generated C file, or the compiler output, as the C file was OK.

Included below is what I did eventually. If the correct method for installation is explained somewhere, then I must have missed it; however, what I went through should not have been necessary.

Best,

Lal

---

Step 1: Command:

python setup_with_libyaml.py install --prefix <path>

Failed on:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/home/lageorge/Tools/include/python2.5 -c ext/_yaml.c -o build/temp.linux-i686-2.5/ext/_yaml.o

In file included from ext/_yaml.c:26: ext/_yaml.h:2:18: yaml.h: No such file or directory

Fix:

redid gcc command with manual -I option added for yaml.h

Result:

Compiled with warnings.

ext/_yaml.c: In function `pyx_f_5_yaml_7CParser_init': ext/_yaml.c:461: warning: passing arg 2 of `yaml_parser_set_input' from incompatible pointer type ext/_yaml.c: In function `pyx_f_5_yaml_8CEmitter_init': ext/_yaml.c:4427: warning: passing arg 2 of `yaml_emitter_set_output' from incompatible pointer

Step 2) Command:

python setup_with_libyaml.py install --prefix <path>

Failed on:

gcc -pthread -shared build/temp.linux-i686-2.5/ext/_yaml.o -lyaml -o build/lib.linux-i686-2.5/_yaml.so

/usr/bin/ld: cannot find -lyaml collect2: ld returned 1 exit

Fix:

manually executed gcc with -L option to gcc. Defining environment variable LD_LIBRARY_PATH did not help

Result:

gcc ran to completion.

Step 3) Command:

python setup_with_libyaml.py install --prefix <path>

Result:

went to completion.