shou chin

Hello, The memory leak was occurring in one complex project using deep learning. We first investigated the problem with tracemalloc that point to the pyfoam with some memory difference report. But We tried to reproduce the leak by extracting the parsing process in a simple project without success. So now we are comment out some code to narrow down the position. The problem is quite complex , we are still monitoring the memory on the server, And I will report again when there is any progress. Sorry...

Hello, The memory leak was occurring in one complex project using deep learning. We first investigated the problem with tracemalloc that point to the pyfoam with some memory difference report. But We try to reproduce the leak by extracting the parsing process in a simple project without success. So now we are comment out some code to narrow down the position. The problem is quite complex , we are still monitoring the memory on the server, And I will report again when there is any progress. Sorry...

Hello, The memory leak was occurring in one complex project using deep learning. We first investigated the problem with tracemalloc that point to the pyfoam with some memory difference report. But We try to reproduce the leak by copy the parsing process in a simple project without success. So now we are comment out some code to narrow down the position. The problem is quite complex , we are still monitoring the memory on the server, And I will report again when there is any progress. Sorry for the...

Hello, The memory leak was occurring in one complex project using deep learning. We first investigating the problem with tracemalloc that point to the pyfoam with some memory difference report. But We try to reproduce the leak by copy the parsing process in a simple project without success. So now we are comment out some code to narrow down the position. The problem is quite complex , we are still monitoring the memory on the server, And I will report again when there is any progress. Sorry for the...

https://stackoverflow.com/questions/6115066/how-much-time-is-the-garbage-collector-using from the above link , it seems python use reference count for simple memory allocation ,but use general gc for circular reference. Is there any circular object allocation in the parsing process? If there are lots of circular object reference that may slow down the gc greatly. Sorry, I made a wrong statement. the total times of our process is about 200000 times instead of 200000 * 5000 . In one step we also parse...

https://stackoverflow.com/questions/6115066/how-much-time-is-the-garbage-collector-using from the above link , it seems python use reference count for simple memory allocation ,but use general gc for circular reference. Is there any circular object allocation in the parsing process? If there are lots of circular object reference that may slow down the gc greatly. Sorry, I made a wrong statement. the total times of our process is about 200000 times instead of 200000 5000 . In one step we also parse...

https://stackoverflow.com/questions/6115066/how-much-time-is-the-garbage-collector-using from the above link , it seems python use reference count for simple memory allocation ,but use general gc for circular reference. Is there any circular object allocation in the parsing process? If there is lots of circular object reference that may slow down the gc greatly. Sorry, I made a mistake statement. the total times of our process is about 200000 times instead of 200000 5000 . In one step we also parse...

Hello, Thank you very much for your investigation . In our app, We have to call the parse file process lots of times, far above 1000 times. Total is about MAX_TIME_STEP*SAVE_INTERVAL_STEP times MAX_TIME_STEP=200000 SAVE_INTERVAL_STEP=5000 So from our test, it will use over 64GB memory finally. From my understanding of python's gc, it is a reference count type gc instead of a general gc like java. That means it will free the memory as far as the reference count was reduce to zero. So I can not understand...