Re: PyYAML hangs when reading a large YAML file
Misha Penkov <[email protected]>
| Newsgroups | gmane.text.yaml.general |
|---|---|
| Message-ID | <CAAyYaWq_Kg-6YwAuP8HGN03n5guarL4U8ibGFwACG=BmpkVAsw@mail.gmail.com> |
On 18 April 2013 21:11, Alexey Zakhlestin <[email protected]> wrote: > > On 18.04.2013, at 15:17, Misha Penkov <[email protected]> wrote: > > I'm trying to load data from a large YAML file (approx. 300MB). PyYAML > hangs when I try to load using this code: > > > > import yaml > > y = yaml.load(open("/tmp/tmp6aJfKz")) > > > > The YAML file itself is valid (it is being output by OpenCV). > > > > Can anyone suggest a way to diagnose this problem? I can provide the > file, but as I've already mentioned, it's fairly large. > > did you check resource-usage? could it be, that Python just grabbed all > RAM it could reach and started to actively use swap? > Thank you for your suggestion. I noticed that RAM usage grew slowly until it consumed almost the entire available RAM (64GB) on my machine. Why is this happening, and how can I get it to load the YAML file properly? ------------------------------------------------------------------------------ Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter _______________________________________________ Yaml-core mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/yaml-core