CPU Utilization > 100% and Memory Utilization > 10GB #183
Comments
|
@ronaldeddings thanks for the feedback in my experience screenpipe uses 8 gb memory (macbook pro m3 max 32 gb) but it can be possible that we have spike usages we will look into this 🙏 PS: atm you can reduce CPU usage (and probably memory) by using dev mode and reducing the you can also use cloud OCR using we provide free cloud usage for a few months I think biggest consumption is OCR atm (could be wrong) we're going to make these settings available in non dev mode soon thanks for the patience 🙏 |
💎 $150 bounty • Screenpi.peSteps to solve:
Thank you for contributing to mediar-ai/screenpipe! |
|
Memory leak is the priority in this issue [addressed to bounty contributors] |
|
Hi @m13v @louis030195 Can you give a little brief on how to reproduce this? It is possible to do it on a windows computer? |
|
I think it the same on all os, just keep running screenpipe, and it will
start accumulating more and more operating memory. First it’s 1gb, in 10
minutes it’s 2gb. In 1 hour it’s 4gb..
…On Fri, Aug 23, 2024 at 1:56 AM Divyam Chandel ***@***.***> wrote:
Hi @m13v <https://github.com/m13v> @louis030195
<https://github.com/louis030195> Can you give a little brief on how to
reproduce this? It is possible to do it on a windows computer?
—
Reply to this email directly, view it on GitHub
<#183 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY62CDFV4QNOWG6SOZI6543ZS32KNAVCNFSM6AAAAABMWRKIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBWGYZDIOJSGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Been running for 21 minutes, when it started it was 300MB only. What am I doing wrong? Because video is not being captured, checked it in the database and the .screenpipe/data as well, no files are being created. Also, the files are not being created via the app that also I checked. I was wondering how could I also change the transcription model to maybe medium. And maybe the problem is it is using orc engine as Tesseract instead of native ocr? |
there is an error with audio |
|
@m13v @chandeldivyam the first task for this issue is to have repeatable measurement of performance, otherwise we're just optimising blindly example to track accuracy & speed of OCR https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-vision/benches/ocr_benchmark.rs |
|
Thats actually very weird thing, in windows or just my computer I am not sure. cpal goes crazy. So, if there is no audio ( i am not on a call or watching a youtube video), basically no system audio. Then there would be no callback from cpal to the stream. It is just for my computer or generally for windows I am not sure. So in a previous project, I had artificially added some vectors without sound. Check the I will start a youtube video in the background for the time being to see the memory issue, but I am very certain about why the current error is there. |
|
So, I started a video i.e. now the output device has audio. We get rid of that error now. |
Checking this, what are your thoughts currently? How could we have a repeatable behavior? Maybe we run individual functions in a proctored env like you did for bench testing of ocr and vision? And plot a graph of memory / other vitals wtr time? That might be the first step for us to find? Also, that would be a good test case for later? |
|
i think we can start by measuring small parts (esp. those that dont need a monitor or audio device so it can run in CI) that likely use lot of compute then if we can find out a repeatable way to measure perf on local computer (e.g. run some command that make screenpipe run for X mins and at the end we know memory & cpu over time, average, spikes, etc.) then if we can find way to simulate monitor and/or audio device in ci would be good also we have this for runtime perf: https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-server/src/resource_monitor.rs but it's not super helpful atm, added feat to log to disk (then you can feed this into chatgpt to analyse perf specifically) |
i increased apple ocr speed by 30% already and increased accuracy by 25% now |
Right, which parts would you suggest we start the benchmarking with first? Also, how to run vision part as well from the terminal with windows native ocr? Currently I am not passing any arugments: The memory is constant at ~300mb, for a minute or so it went to around 600mb but came down again (whatever it was, it must have been dropped after the process). Been there for last ~30 minutes. With no error in the console. |
|
After about 60 iterations, windows sent it to efficiency mode. Using around 45mb memory Currently at 87 iteration, still same memory utilizaiton. @m13v @louis030195 Audio files are being recorded, also transcriptions are being inserted to the db |
|
check args
|
looking at apple Instruments, it seems the "chunking" part of screenpipe uses ton of CPU will push a benchmark for this |
|
@chandeldivyam also if you can fix the windows version of this: https://github.com/mediar-ai/screenpipe/blob/main/screenpipe-vision/benches/ocr_benchmark.rs that would be great
to run |
Have been running with -> It gradually increased from 300mb to 1500mb but then suddenly came back to 300mb again. GPU VRAM is ~3 GB from screenpipe.
Checking this, will raise a PR |
Yes, similar observation in windows. Can not pinpoint what it is but every 30 seconds when there are these logs: There is a spike in CPU utilization. |
|
|
Ran the screenpipe Initially 400mb -> 1500mb (for decent amount of time) -> 400mb (again for a decent amount of time) But, there is no image being captured. Checked the db and storage location. What could the issue be? There is no error log either. Neither is image capture running from the app. |
|
@chandeldivyam never saw this issue before EXCEPT when windows defender decide to delete screenpipe or similar so there is no logs error or something related to vision? you can try adding |
will check this |
Maybe this could be the reason? I can see entries in In Maybe some permission issue? |
|
btw for mac one of the memory issue i suspect is:
to confirm |
can you try to run this maybe https://github.com/nashaofu/xcap/blob/master/examples/window.rs |
|
Ran the terminal as administrator, didn't get the error.
I think this should work because we are getting screen captures. As we can are getting the ocr_text table filled. But let me check. |
|
so first source of leaks: https://github.com/RustAudio/cpal/pull/894/files second is xcap: i suggest we suggest we switch to |
|
|
trying a hack now to fix with xcap |
|
update: any help to switch to this: https://github.com/mediar-ai/scap would be good we need to impl: |
|
Got an issue, which was making my RAM go crazy. After changes, recording the screen for the last 40 minutes and ram didn't move an inch.
Till the time it goes there, my system basically crashed. For around len of 10, it was 5GB, so it needed 50GB to reach 100. Changed it to max = 10 and have been running the recording at --fps = 1, its been going on for 50 minutes and no memory increase. Checked everywhere from my understanding, We should do two things now ->
|
|
|
Perfect, create a pull request, seems like you've solved it! Congrats!
…On Fri, Aug 23, 2024 at 12:52 PM Divyam Chandel ***@***.***> wrote:
[2024-08-23T19:50:31Z INFO screenpipe_server::video] Starting FFmpeg process for file: C:\Users\ABC\.screenpipe\data\2024-08-23_19-50-31.mp4
[2024-08-23T19:50:31Z INFO screenpipe_server::resource_monitor] Runtime: 3050s, Total Memory: 1% (0.30 GB / 23.94 GB), Total CPU: 55%
[2024-08-23T19:50:51Z INFO screenpipe_server::resource_monitor] Runtime: 3070s, Total Memory: 1% (0.19 GB / 23.94 GB), Total CPU: 50%
[2024-08-23T19:51:01Z INFO screenpipe_server::resource_monitor] Runtime: 3080s, Total Memory: 1% (0.30 GB / 23.94 GB), Total CPU: 46%
[2024-08-23T19:51:11Z INFO screenpipe_server::resource_monitor] Runtime: 3090s, Total Memory: 2% (0.50 GB / 23.94 GB), Total CPU: 49%
[2024-08-23T19:51:21Z INFO screenpipe_server::resource_monitor] Runtime: 3100s, Total Memory: 1% (0.20 GB / 23.94 GB), Total CPU: 55%
image.png (view on web)
<https://github.com/user-attachments/assets/8554b997-c834-45c9-94bb-f20898fabe06>
—
Reply to this email directly, view it on GitHub
<#183 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY62CDAPCAJIAS6FGZ2LBDDZS6HG5AVCNFSM6AAAAABMWRKIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBXG4ZDCMRSGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@chandeldivyam good job, actually this was my bad when trying to solve this issue by using data structures that don't grow infinitely (e.g. i released new version with this fix in app & brew now this does not solve the original issue (my memory still growing forever) though which is essentially xcap and screencapture leaking memory due to objects not being released in 
i suggest we focus on switching to
regarding macos audio output the leak seems acceptable for now because we don't call so frequently the function (once every 30s, while the Windows:all() is probably called >5 times every frame per monitor) |
|
@louis030195 Yes, we should also first create a mechanism to benchmark. I feel this because xcap issue must be with screencapturekit. Because I ran screenpipe (on windows) without audio for hours, the memory didn't move. It was same / lower than where it started in first 30 seconds. |
|
Maybe just something like a plot against time while video capture? Current reporting benchmark itself (which we see in the terminal) plotted over time? This could potentially help us understand if something is changing. |
xcap does not use screencaptuekit (mac) they use old apple api screencapturekit is the new api for mac used in scap using scap would also solve #63 (about 3-4 linux users cannot use screenpipe because of this) we log to files resource usage in here: you need to add and i did a google colab to create charts out of this data: https://colab.research.google.com/drive/1zELlGdzGdjChWKikSqZTHekm5XRxY-1r?usp=sharing in the past i was working in observability team to track billion of devices performance with promotheus + grafana but i dont think this is good for consumer things, we just have to write metrics ourselves ideally we should use this well (although this is more for logging): i think we should take inspiration on how they use it: https://github.com/search?q=repo:huggingface/candle%20span&type=code |
|
Great, I think we are well researched then, that there is tangible benefit of moving from xcap -> scap Let me look into both xcap and scap and try to migrate us to scap. Things I am wondering right now:
Let me look into this section and will update. |
|
@chandeldivyam good! i will focus on "visible/focus window and app name" for macos now and push here https://github.com/mediar-ai/scap once this is usable on macos/linux/windows lets replace xcap in screenpipe |
|
Perfect |
|
I went through the screenpipe code as well. We should ideally get the response from scap and transform it, in a back compatible format.
The functions capturing and validating the images would change and we can transform the new output by scap into our impl, that is something which I was thinking. |
this was not the issue
|
|
the main issue is solved by upgrading to the bounty is still live to solve the 2nd memory leak here: https://github.com/louis030195/cpal-d this is less a problem because we call infrequently so would cause memory issue only after having ran screenpipe for days |
|
increasing the bounty /bounty 150 e.g. make the https://github.com/louis030195/cpal-d not leaking |
|
#236 moving issue here (bounty 150) |
I'm experiencing high CPU and memory utilization from the screenpipe process
Under settings, application is showing status "healthy"
Any updates would be greatly appreciated so that I can run screenpipe with other memory intensive applications at the same time
The text was updated successfully, but these errors were encountered: