Rechecking of every torrent happens every time I open qBt after unlocking a Bitlocker drive

[github-account1111]

qBittorrent & operating system versions

4.4.0
Windows 10 1809 x64

What is the problem?

Every time I open qBt after unlocking a Bitlocker-locked drive, all the torrents on that drive have to be rechecked which

1. Takes hours and
2. Slows qBt down to a crawl

Steps to reproduce

1. Open qBt with some torrents on a locked Bitlocker-locked drive (those torrents will be marked as Bitlocker-locked),
2. Close qBt
3. Unlock the drive
4. Re-open qBt

Additional context

No response

Log(s) & preferences file(s)

No response

[ClandestineClout]

This is a problem with qbittorrent overall. About once a week it has to re-check 25 terabytes of data which can take up to a week. I rarely if ever have every torrent fully checked.

Actually it's one of the things that's making me consider dropping all torrenting and going back to streaming services.

I just can't wrap my head around how we haven't figured out how to overcome this problem. Quite frankly, it's the giant elephant in the room nobody wants to talk about.

[shama84]

Hi,
So you get this after opening QBT when the drive is still locked, correct? This sounds pretty normal for me and this is not limited to QBT in my usage (DBs etc). But why is the drive not unlocked? The OS drive is unlocked at boot, but your can auto unlock data disk too
https://docs.microsoft.com/en-us/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps
https://www.tenforums.com/tutorials/37662-turn-off-auto-unlock-bitlocker-drive-windows-10-a.html

[shama84]

This is a problem with qbittorrent overall. About once a week it has to re-check 25 terabytes of data which can take up to a week. I rarely if ever have every torrent fully checked.

Actually it's one of the things that's making me consider dropping all torrenting and going back to streaming services.

I just can't wrap my head around how we haven't figured out how to overcome this problem. Quite frankly, it's the giant elephant in the room nobody wants to talk about.

This is not something I experienced with less torrents how ever (<1000)

[ClandestineClout]

I'm not sure that's relevant. QB has always been finnicky with needing to recheck torrents. I only recently started downloading a lot more. Only a few months ago I had a few dozen torrents. It has always been a problem. There are just too many situations that lead it to not make a clean exit, or whatever other reason. Sometimes it will do a super fast recheck, (about 1 in 10 times this happens) but most of the time it will go pixel by pixel... Could be an hour for a single torrent to get checked on a slow drive. I mean at this point it's costing me more to manage torrents than it would to just buy services. This is getting to be a little ridiculous.

…

On Mon, Jan 24, 2022, 9:02 AM shama84, ***@***.***> wrote: This is a problem with qbittorrent overall. About once a week it has to re-check 25 terabytes of data which can take up to a week. I rarely if ever have every torrent fully checked. Actually it's one of the things that's making me consider dropping all torrenting and going back to streaming services. I just can't wrap my head around how we haven't figured out how to overcome this problem. Quite frankly, it's the giant elephant in the room nobody wants to talk about. This is not something I experienced with less torrents how ever (<1000) — Reply to this email directly, view it on GitHub <#16240 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVS6NSZRHNSF7ETSH6ZUGQTUXVLXLANCNFSM5MTZYNBA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[shama84]

One of the big variable in a recheck is the torrent pieces size (check general>Pieces). The bigger they are the fast it goes.
Now with so much torrents the IOPS pressure may be high on your system and variables can lead to slowness. Example with a non-aligned RAID system or volume with big cluster size (16KB or more): this add unnecessary IOPS for each read/write. And there are a lot for torrenting...The storage subsystem is a big topic but can have a big influence.
BTW (I'm still thinking you have good storage subsystem) have you checked the caching setup of volume hosting the torrents? This may be a source of problem if you have to recheck all of them often

[ClandestineClout]

Just curious, how long does it take you to recheck 25tb of data? What is a normal amount of time for that for a USB drive?

[shama84]

Are you on USB Drive with 25TB?
Is your USB drive UASP compatible?

[ClandestineClout]

Actually multiple external HDDs connected via a powered USB hub. Running an an old gaming laptop that only has USB 2.0, which probably makes a difference. Is that what this is all about? The usb 2 bottleneck?! Why didn't I see that before?

Edit... No. It can't be... Because it's still obnoxiously slow to check on the internal SSD (via SATA), maybe only a little bit faster. Even that's exaggerating.

[shama84]

If such setup suits your needs no problem of course. About the "before" I guess you mean with older version of QBT up to the 4.3.9? This rings a bell #16043

[github-account1111]

So you get this after opening QBT when the drive is still locked, correct? This sounds pretty normal for me and this is not limited to QBT in my usage (DBs etc).

No, this is after unlocking the drive and reopening qBt.

But why is the drive not unlocked? The OS drive is unlocked at boot, but your can auto unlock data disk too

This is on an external drive.

This is really frustrating.
It's needlessly wearing off and killing my drive (sitting at 100% in task manager for literal hours), rendering it unusable while the checks are happening, tanking my CPU performance (30-40% in task manager) and for what?
I don't actually get anything out of this.

[shama84]

No, this is after unlocking the drive and reopening qBt.

Once the file is marked for recheck this won't change up to action completion

This is on an external drive.

Try the auto-unlock link it will ease your life :). Or reconsider the bitlocker necessity ?

This is really frustrating.
It's needlessly wearing off and killing my drive (sitting at 100% in task manager for literal hours), rendering it unusable while the checks are happening, tanking my CPU performance (30-40% in task manager) and for what?
I don't actually get anything out of this.

This is indeed a frustrating situation but it doesn't involve really QBT. From my understanding the behaviour is expected. You have your reason to use bitlocker but it comes with this drawback. A script to launch QBT can also ease your life (unlock the drive and open QBT after).

[github-account1111]

Once the file is marked for recheck this won't change up to action completion

I think this is a problem.
Why won't it?
It can obviously detect the reason it can't resume - it even states something along the lines of "Can't resume due to Bitlocker" in the file status, as I noted in reproduction steps.

Try the auto-unlock link it will ease your life

Doesn't work for my threat model.

Or reconsider the bitlocker necessity ?

As in, not encrypt at all?
If so, no.
Encryption is a must today.
I shouldn't have to give up security just because some app can't deal with encryption.
Instead the app should be able to deal with it.

This is indeed a frustrating situation but it doesn't involve really QBT.

It does though.
The problem is that qBt doesn't respect the fact that the files are still intact even though it knows that they are since it clearly states so in the file status.

Come to think of it, it shouldn't auto-recheck at all ever.
I shouldn't have to wait for ages every time I open qBt without having my external drive connected.
I'm on a laptop.
There are many instances where I don't even have my drive at hand but still need to use qBt.
I've no idea why this is an expected behavior.
What's it intended to solve?

[shama84]

I think this is a problem.
Why won't it?
It can obviously detect the reason it can't resume - it even states something along the lines of "Can't resume due to Bitlocker" in the file status, as I noted in reproduction steps.

At this layer the app is not aware of the reason, file is not accessible and the flag is set for recheck.

As in, not encrypt at all?
If so, no.
Encryption is a must today.
I shouldn't have to give up security just because some app can't deal with encryption.
Instead the app should be able to deal with it.

The Bitlocker encryption protects the USB disk when locked, so the data it contains. During the unlock (QBT usage) the disk is accessible for anything running on the operating system. Protecting your IP or encrypt the network traffic is a different matter of course.The app cannot deal with Bitlocker because only the OS is aware of the encryption.

What's it intended to solve?

As stated above QBT cannot deal with Bitlocker, not on the same OSI layer. The fact QBT marks the file for recheck when not available is annoying in your situation but so is it. It can perfectly download the file again if the location is not accessible but it will be worst, we can agree.

There are many instances where I don't even have my drive at hand but still need to use qBt.

I think such usage will never work, the client needs access to loaded torrents all the time. You can pause all torrents before closing the app each time (only guess), or you can use multiple torrent client instance (better?).
I get ride of such by installing a dedicated hardware for QBT torrent usage, always on, that could be simply an RPI4 or any spare device you have. The solution will be different for everyone but there are some parameter you cannot change

[github-account1111]

At this layer the app is not aware of the reason, file is not accessible and the flag is set for recheck.

Which is the problem.
It should be aware.

The Bitlocker encryption protects the USB disk when locked, so the data it contains. During the unlock (QBT usage) the disk is accessible for anything running on the operating system. Protecting your IP or encrypt the network traffic is a different matter of course.

I know how Bitlocker works, not sure why you're telling me this.

The app cannot deal with Bitlocker because only the OS is aware of the encryption.

So every other app can except qBt.
Kinda weird, don't you think?

As stated above QBT cannot deal with Bitlocker, not on the same OSI layer. The fact QBT marks the file for recheck when not available is annoying in your situation but so is it. It can perfectly download the file again if the location is not accessible but it will be worst, we can agree.

Not sure what you're trying to say.
The whole point is it cannot "perfectly download the file again."

I think such usage will never work, the client needs access to loaded torrents all the time.

Why won't it?
I gave a literal solution in my previous comment.
Make the rechecking triggers less overzealous.
In fact, one of the devs in #15747 (comment) even said they've been trying to do just that, but that there may be cases where it's still unnecessarily triggered and asked to report such cases which is exactly what I'm doing in this issue.

You can pause all torrents before closing the app each time (only guess), or you can use multiple torrent client instance (better?).

That is a horrible workaround.
Having to manually pause every torrent that isn't on the system drive every time I close qBt?
That's what software is for.
It should take care of this automatically.

I get ride of such by installing a dedicated hardware for QBT torrent usage, always on, that could be simply an RPI4 or any spare device you have. The solution will be different for everyone but there are some parameter you cannot change

Again, not sure what you're trying to say.
I really think you misunderstood the issue.

[shama84]

Indeed I miss something here, this became obvious after a second reading :/

[ghost]

Will be fixed in 4.5.0 when used in conjuction with latest libtorrent.