python - Why is this lxml.etree.HTMLPullParser leaking memory? -


i'm trying use lxml's htmlpullparser on linux mint i'm finding memory usage keeps increasing , i'm not sure why. here's test code:

# -*- coding: utf-8 -*- __future__ import division, absolute_import, print_function, unicode_literals import lxml.etree import resource io import default_buffer_size  _ in xrange(1000): open('stackoverflow.html', 'r') f:     parser = lxml.etree.htmlpullparser()     while true:         buf = f.read(default_buffer_size)         if not buf: break         parser.feed(buf)     parser.close()      # print memory usage     print((resource.getrusage(resource.rusage_self)[2] * resource.getpagesize())/1000000.0)

stackoverflow.html homepage of stackoverflow i've saved in same folder python script. i've tried adding explicit deletes , clears far nothing has worked. doing wrong?

elements constructed parsers leaking, , can't see api contract violation in code that's causing it. since objects survive manual garbage collection run gc.collect(), best bet try different parsing strategy workaround.

to see root cause, used memory exploration module objgraph , installed xdot view graphs created.

before running code, ran:

in [3]: import objgraph  in [4]: objgraph.show_growth()

after running code, ran:

in [6]: objgraph.show_growth() tuple                  1616      +147 _element                146      +146 list                   1100       +24 wrapper_descriptor     1423       +15 weakref                1155        +6 getset_descriptor       677        +4 dict                   2777        +4 member_descriptor       315        +3 method_descriptor       891        +2 _tempstore                2        +1  in [7]: import random  in [8]: objgraph.show_chain(    ...: objgraph.find_backref_chain(    ...: random.choice(objgraph.by_type('_element')), objgraph.is_proper_module)) graph written /tmp/objgraph-bfuwa9.dot (8 nodes) spawning graph viewer (xdot)

note: numbers might different see depending on webpage viewed.


Comments

Post a Comment

Popular posts from this blog

php - failed to open stream: HTTP request failed! HTTP/1.0 400 Bad Request -

this code i'm using find latitude , longitude of place: function coords($address){ echo $url="http://maps.googleapis.com/maps/api/geocode/json?address=".$address; $json = file_get_contents(urlencode($url)); $data = json_decode($json, true); print_r($data['results'][0]['geometry']['location']['lat']); print_r($data['results'][0]['geometry']['location']['lng']); } however returns warning: failed open stream: http request failed! http/1.0 400 bad request if use above code in simple procedure works fine not in function. note: have tried curl_init()... method produced same result? instead of encoding whole url encode address part. following code work fine $add='jamshoro phase 2'; $url="http://maps.googleapis.com/maps/api/geocode/json?address=".urlencode($add); $json = file_get_contents($url); $data = json_decode($json, true); print_r($data['r
Read more

java - How to filter a backspace keyboard input -

i have code gives message when key number pressed: txtvalueofclause.addeventfilter(keyevent.key_typed, new eventhandler<keyevent>() { @override public void handle(keyevent t) { char ar[] = t.getcharacter().tochararray(); char ch = ar[t.getcharacter().tochararray().length - 1]; if (!(ch >= '0' && ch <= '9')) { system.out.println("the char entered not number"); t.consume(); } } }); now if press on wrong button , press backspace remove it, error message. how can add backspace input if statement? i have found answer: txtvalueofclause.addeventfilter(keyevent.key_typed, new eventhandler<keyevent>() { @override public void handle(keyevent t) { char ar[] = t.getcharacter().tochararray(); char ch = ar[t.getcharacter().tochar
Read more

java - Show Soft Keyboard when EditText Appears -

i have code in android app, alert dialog , keyboard, , when dialog shown want keyboard appear. not , not sure why? here code: final alertdialog.builder alert = new alertdialog.builder(this); alert.settitle("title"); alert.setmessage("message"); final edittext input = new edittext(this); input.setinputtype(inputtype.type_class_number); alert.setview(input); input.setonfocuschangelistener(new view.onfocuschangelistener() { @override public void onfocuschange(view v, boolean hasfocus) { if (hasfocus) { log.i(tag, "in focus"); inputmethodmanager inputmethodmanager = (inputmethodmanager) getsystemservice(context.input_method_service); inputmethodmanager.showsoftinput(input, inputmethodmanager.show_implicit); } } }); but keyboard not appear though edit text in focus, , tell appear why, doesn't come up? thanks in advance.
Read more