/hydrus/ - Hydrus Network

Archive for bug reports, feature requests, and other discussion for the hydrus network.

Catalog Archive
Name
Options
Subject
Message

Max message length: 8001

Files

Max file size: 32.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and postings)

Misc

Remember to follow the Rules

The backup domain is located at 8chan.se. .cc is a third fallback. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 2.0.

Maintenance complete. Other systems remain to be tested and updates will come as they are ready.
Ghost Screen: Especial de Halloween primer sรกbado
Comienza a las 0UTC
El Hilo


8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

(32.04 KB 480x360 r1nn-tp26KE.jpg)

Version 425 Anonymous 01/13/2021 (Wed) 22:34:36 Id: 6d169b No. 15109 [Reply]
https://www.youtube.com/watch?v=r1nn-tp26KE windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Linux.-.Executable.tar.gz I had a good week. I optimised and fixed several core systems. faster I messed up last week with one autocomplete query, and as a result, when searching the PTR in 'all known files', which typically happens in the 'manage tags' dialog, all queries had 2-6 seconds lag! I figured out what went wrong, and now autocomplete should be working fast everywhere. My test situation went from 2.5 seconds to 58ms! Sorry for the trouble here, this was driving me nuts as well. I also worked on tag processing. Thank you to the users who have sent in profiles and other info since the display cache came in. A great deal of overhead and inefficient is reduced, so tag processing should be faster for almost all situations.

Message too long. Click here to view full text.

2 posts omitted.
Maybe its time we moved to a generic forum. I hate the idea but we've been jumping from 1 chan to the next so quick lately. I wonder if they make accountless forum.
>>15111 They sent a message to all BOs about it, and there are some (I think new) little text banners here and there saying the TOR link on the main page and so on. My guess is relations with their host aren't super great, and with all the other deplatforming going on, they are just waiting for the email now. Sounds like the situation sucks. I don't browse here any more either other than /hydrus/, and I should have moved a while ago, I was just putting it off. Although, funnily enough, having said I would 'lock' the board, I then couldn't find that option in the admin panel. I'll have another look today once I am caught up with the main threads, but if you just can't do that, I'll make a sticky with an announced deletion date. Making an archive of the Q&A thread is probably worthwhile. >>15114 I am not sure. I considered trying for 8chan.moe or somewhere in the webring, since that is where I browse these days personally, and they are opening up board creation in more places. But I think I'll just sit at endchan for now. It is a nice quiet corner. I have never been a comfortable board owner, and never good about running and checking a board for spam etc…, so I'd probably be happier just being a janny on a board run by others, which is how the hydrus discord works, although I wouldn't want to ask people to do hotpocket work unless they are keen. >>15121 Yeah, jumping around has been frustrating. My feeling is we will get another wave of hosts setting new rules over the next 1-3 months and taking out wrongthink and then things will settle again. If you are comfortable in a more normie-friendly experience, the discord is at https://discord.gg/wPHPCUZ The plan to move to Endchan as primary board has changed. Codexx here on 8chan.moe kindly offered to host me on /t/, so I will now be maintaining a Hydrus Network General thread there.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:44:21.

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #4 Anonymous Board volunteer 04/16/2022 (Sat) 17:14:57 No. 17601 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
659 posts and 101 images omitted.
>>18216 I haven't done the big change on the Linux side just, so this may be placebo, but I've also done some other video/mpv work recently that may have reduced some lag for you. As I asked in the v499 changelog, some Linux users tested out the new python-mpv wrapper, and they said it works well, so I will be updating it in the Linux build for v500. I think you'll still see 1.109 for now, but when libmpv1 either in my build or your system (through apt-get) gets updated, the new python-mpv should handle it well. >>18217 Damn, thank you for this report. I am sorry for the trouble and will check this out. I did work on that loop-checking logic recently, I thought to improve things and help bunch stuff together for the janitors, so perhaps there is a new bug in there. >>18218 You know I was reading this recently https://support.nordvpn.com/General-info/Features/1845333902/What-is-Meshnet.htm , which as I understand it is basically user-friendly hamachi. If the other vpn providers soon offer the same tech, and consumers get access to easy virtual local networks, some of our hydrus-specific pains is the ass may evaporate. All the shit about setting up proxy forwards and self-signed ssl certificates and things disappears if the secure network side of things is handled by a vpn provider and your phone thinks it is talking to 192.168.x.x. That said, if you are already on the same network, your best bet is just to boot up the Client API and try out Hydrus Web. If you already have 192.168.x.x:45869 visible and just want to sit on the couch with your tablet and browse things, this is possible right now without too much strife: https://hydrusnetwork.github.io/hydrus/client_api.html >>18219 Probably early next year. I'm going to start with the very easy case of jpeg+png exact pixel dupes (keep the jpeg), just to get a skeleton of an auto-dupe-filter started, and then flesh it out with cleverer rules from there. Everything will be completely optional, default off. My dream is that in five years we have all sorts of tech making weighted decisions about dupes, and 95% of stuff is handled without you ever seeing it, including immediately on import. >>18220 Maybe. We've played with this tech before. A legacy system called 'file lookup scripts' allows you to do this on a per-file basis in manage tags, but I think you have to turn it on under options->tag suggestions. If I do it, it would hook into the slow-burn system I describe here >>18258 since the same bandwidth and workflow concerns apply.

Message too long. Click here to view full text.

>>18221 >>18222 I am very sorry, I think you have overwritten your database, yes. You still have your actual media, the jpegs and so on, but your archive record, known urls, the session and subscriptions, all the stuff you see in the UI, has probably been overwritten. You probably already have, but I recommend you have a very good think about if there is any chance an old copy of your database might sit in your recycle bin or an old usb stick somewhere. If there is one, we can probably stitch a rollback together. If not, you may be looking at starting a new client and then importing your old install_dir/db/client_files structure. You would be looking at the 256 'fxx' subdirectories in that folder. For the backup situation, I am sorry. I know how it feels to lose your stuff. Although it is a shame, it is usually emotional kicks like this that ultimately give you the motivation to figure out a good backup solution in future. You don't want to lose shit like this again, so you put the time and money in in future. You don't have to do it today, but if you have a planner, make sure in a week or two that you come back and plan a proper backup routine. Not just for hydrus, but your documents and writing, any art or programming you do, anything digital, so if your machine blows up or your place burns down you still have a copy on a USB stick in your backpack, whatever works for you. My backup help is here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up And a special message for you is here: https://hydrusnetwork.github.io/hydrus/after_disaster.html I'm happy to help with recovery or setting up your future backup, just let me know. >>18225 Thank you for this report. I will investigate this. Unfortunately, crashes (as in the program halts and exits immediately) will not record any information in the log, but if the program simply halts and does not respond, there may be something in the log as >>18236 says. As you are a Win 7 user and Qt6 will not work for you, you may be looking at the option of running from source soon. The users in this chain >>18213 found that running from source reduced their crashes significantly, so if your problem is similar, this may be something to think about.

Message too long. Click here to view full text.

New General here: >>>/t/9763 This one should be migrated to /hydrus/ soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #3 Anonymous Board volunteer 12/01/2021 (Wed) 23:12:07 No. 16965 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
632 posts and 156 images omitted.
https://www.youtube.com/watch?v=R1t6iNG28zI windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Linux.-.Executable.tar.gz I had a good week. Notes now display on the media viewer. notes Notes have always been a slightly hidden system, a bit like ratings were. Today is a step forward to exposing them. Any file that has notes (you can start adding notes to a file by hitting manage->notes on their right-click menu) will now show them in the media viewer, just below the top-right hover window. They get their own hover window too, if you mouse over them. If you click on a particular note, the 'edit notes' dialog opens on it. This is a first version, and a little ugly, but I'm happy we now have something I can iterate on in future. If you are a big notes person, please let me know how it works best and worst for you. If you have unusual font style, size, or colour, let me know if it goes crazy or sizes too short or tall.

Message too long. Click here to view full text.

(608.93 KB 1366x768 Screenshot_20220406_182040.png)

(765.98 KB 1366x768 Screenshot_20220406_182326.png)

(516.64 KB 1366x768 Screenshot_20220406_182538.png)

(89.35 KB 1023x1023 clapping.gif)

>>17598 Marvelous!!! It is a nice touch that when more than one note is present and clicking on that note, the Note dialog will open showing that precise tab on foreground. Something that at least for me is annoying is that Notes will open with the scroll at the end and not at the beginning as should be expected. See pic 3.
Hey, unfortunately 8chan has had some posting trouble and we have lost a week or so of posts. Since this thread is bumplocked, I was going to make a new thread for 481 anyway, so Hydrus General #4 is here: >>>/t/8151 This thread will be moved to the /hydrus/ archive soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #2 Anonymous Board volunteer 04/20/2021 (Tue) 22:48:35 No. 15850 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
1111 posts and 210 images omitted.
Out of curiosity, is it possible to browse through all the tags present in the public tag repository? For instance finding out how many instances of tag A exists and perhaps how many files with tag A also has tag B
I had trouble posting this last night. I hadn't realised how long the thread was--I will make a new general today for the release post! I had a good week. I managed to add support for embedded ICCs, which will improve some images' colours, and overhauled how files are deleted from the client and server so the system is smoother and more reliable. The release should be as normal tomorrow.
>>16962 Unfortunately the sheer number of tags--tens of millions--make it difficult to 'browse' in UI, so your best shot for now is to access the SQLite database files manually and run your own statistical queries. I can help you with that if you like. Another simple option is just to run some basic searches on 'all known files'. This is advanced, so be careful of running some super CPU heavy searches, but turn on help->advanced mode, and then change your search page from 'my files' to 'all known files' and the tag domain from 'all known tags' to 'PTR'. Then do some searches, you'll see real tag counts and get ghost results of non-local files if you run a search. It is borked developer-mode stuff mostly for mine or the janitors' use, but you can learn some interesting things. New Thread >>6469

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #1 Anonymous Board volunteer 01/20/2021 (Wed) 04:21:14 No. 15143 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
703 posts and 145 images omitted.
>>15842 I usually do everything using bash, so I have this script I named hydrupdate: #!/usr/bin/env bash test -z "$1" && exit INSTALLDIR='/your/install/dir/here/' test ! -d "$INSTALLDIR" && echo 'The provided install dir does not exist or is not a directory' && exit tar xf "$1" -C "$INSTALLDIR" --strip-components=1 I can then call the script with hydrupdate /path/to/Hydrus.Network.<release>.-.Linux.-.Executable.tar.gz.
New thread for v436 >>3626 !
>>15839 Thank you, I will! These questions come up enough, and I sometimes forget the details myself, that there should be an official write-up somewhere. >>15840 The other guys are also correct here, but if you are using one of the 'extract' releases, basically you just drag and drop the contents of the archive onto your install directory. Longer explanation here: https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#updating >>15843 Thanks, I will have a look. I guess the third click is getting sent by Qt as a second doubleclick event. I'll see if I should be terminating the click-chain or whatever it is called when I eat the initial double-click. >>15846 Yep, check options->sort-collect. The UI is debug-tier bullshit, I apologise, but if you add some new sort types, any namespaces in those sorts will be added in your collect dropdown.

(1.59 MB 847x860 1425150841112.gif)

Remote backups with Hydrus Anonymous 01/20/2020 (Mon) 13:27:14 Id: c2f342 No. 13543 [Reply]
Hello everybody, I've been wondering how some of other Hydrus Network uses handle backups for their files. My current setup is like thus: I make weekly local backups of my database, which are then sent to a NAS through UrBackup, From there, I've been using Duplicati with cheap S3-compatible remote object storage. Now, supposedly Duplicati performs deduplication before uploading, however it still seems like it takes its sweet time on the SQLite database files themselves, so I'm wondering if their format just doesn't lend itself to be deduplicated adequately or if there is some other issue. I don't use password protection (although from what I've seen, it doesn't actually encrypt anything by itself so it shouldn't make much of a difference either way) So, what other methods do you use to backup your Hydrus files remotely? Ideally it should be with deduplication because although space is not at a premium for me, my upload speeds are very limited.
4 posts and 1 image omitted.
>what other methods do you use to backup your Hydrus files remotely? run a hydrus client on a server and set it as a remote file repository, then you can tell your local instance to upload to it.
>>15078 I'm the anon from the OP, the main issue with this approach is that without deduplication of some kind, you'll end up having to send your (rather sizable) database file with every update all over again every time you rsync with the remote folder. As I mentioned last year, storage space isn't so much a concern as it is upload speed, as even with a mild recent improvement they still top around 2Mbps. I had to stop using Duplicati as it was incredibly fragile, I'm now using borg and rclone to send my local repo to a Backblaze B2 bucket (even cheaper than my previous option). Borg handles the deduplication and encryption.
>>15120 With something like borgbackup/rsync you can do it in a way that it only updates changed files I believe. So you're not backing up the entire db over and over, only changes.

Bugs Thread hydrus_dev 02/06/2019 (Wed) 01:35:09 Id: 2b149c No. 11542 [Reply] [Last]
BUGS THREAD
500 posts and 1 image omitted.
This thread is now archived at https://archive.is/cq5Tc . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/9.html , thank you! This plan has changed. The main imageboard location for hydrus is now a General on >>>/t/ here on 8chan.moe.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:40:11.
>>14683 >If you do have it, is MPV the player under options->media? If your hydrus does not have libmpv access, or is failing to import it, it will fall back to my native viewer, which has no audio support. Fixed. Thanks a lot. I'm new to Hydrus. Using Linux MX. Before I was using Tagspaces which is a sluggish piece of trash. Looking for a more suitable alternative I bumped into Hydrus; so far I'm very pleased by its performance and rich settings.
>>15118 Is there a way to get the blacklist back to working? There's usually a cookie for you blacklist on gelbooru but for some reason the downloader doesn't use it and stopped working for me a month or so ago. I can't remember if it was something I did or if the site changed or what.

(4.66 MB 4000x2715 shutterstock_89245327.jpg)

Parsing scripts Anonymous 11/14/2016 (Mon) 18:14:13 Id: f047d8 No. 4475 [Reply] [Last]
How about a thread for discussing/creating/sharing parsing scripts? I made one for md5 lookup on e621.net (actually I just modified Hydrus_dev's danbooru script). Let me know if I did anything wrong with it, I'm pretty cluelessโ€ฆ but it seems to work fine.
[32, "e621 md5", 1, ["http://e621.net/post/show", 0, 1, 1, "md5", {}, [[30, 1, ["we got sent back to main gallery page -- title test", 8, [27, 1, [[["head", {}, 0], ["title", {}, 0]], null]], [true, true, "Image List"]]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-general"}, null], ["a", {}, 1]], null]], ""]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-copyright"}, null], ["a", {}, 1]], null]], "series"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-artist"}, null], ["a", {}, 1]], null]], "creator"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-character"}, null], ["a", {}, 1]], null]], "character"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-species"}, null], ["a", {}, 1]], null]], "species"]], [30, 1, ["we got sent back to main gallery page -- page links exist", 8, [27, 1, [[["div", {}, null]], "class"]], [true, true, "pagination"]]]]]]
86 posts and 1 image omitted.
I'm not sure if GUGs can make these, but anyone have a module for setting up Youtube subscriptions?
>>14782 They're not hiding lolis. I don't understand why I keep hearing this. Did you check the mature content option in settings and clear your account blacklist? Do you have an account in the first place?
Can someone help me understand what parsing scripts are for, and how to use them? Are they to improve the amount of tags that are found for images? Like a reverse search?

Q&A Thread: For simple questions that don't need their own thread Anonymous 09/12/2020 (Sat) 23:55:46 Id: ff6d52 No. 14731 [Reply] [Last]
Here you can ask questions so that the board is not clogged with small threads. Old thread >>12641
158 posts and 1 image omitted.
>>15074 Any site that doesn't operate like a normal booru is tricky. Sometimes you just have to babysit them. I assume you are downloading through nitter, right? I think the twitter API that nitter hooks into does not deliver tweets beyond about 3,000 tweets into the past, so if you are hitting that limit somehow, or if nitter has its own limit, that may explain it. I have been getting reports recently that nitter has been giving '429 - server rources limited' or similar recently as well. This may be related, I am not sure. >>15088 I am not sure if the existing pixiv login script works right now, it has always been temperamental. If not, I think you'll have to use Hydrus Companion or similar to log in by copying your browser cookies to hydrus. Then pixiv will think your hydrus is you and it'll have access to private stuff. >>15095 Support here is not yet great. If you are importing from hard drive, they should import in filename order. If you are importing from a site, it will come in the order the site's parser pulls the URLs, which is usually good, sometimes reversed. The main problem is hydrus currently has integer timestamp for import time, so if two files are imported in the same second, they will sort randomly when you sort by import time. If this is what is hitting you, I apologise. At some point there will be a conversion to sub-second times, although a retroactive fix would likely be impossible. I am hoping the eventual file alternates work will provide a nicer system than page tags. I agree they are clunky.
>>15115 >>15116 This is some developer bullshit unfortunately. You don't need mpv, but 'libmpv1'. You should be able to get it with apt.
This thread is now archived at https://archive.is/0WoQk . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/10.html , thank you! This plan has changed. The main imageboard location for hydrus is now a General on >>>/t/ here on 8chan.moe.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:36:35.

Yiff.party is Dead... Any plans to help rip patreon or SubscribeStar? Anonymous 11/15/2020 (Sun) 19:49:17 Id: 0dfb73 No. 14929 [Reply]
Kemono.party is not working right now and seems to be very unstable. A good idea would be to get patreon/subscribestar downloaders working. Currently hydrus will try to look at patreon pages then get booted out, a possible code reference for how to rip it might be here https://github.com/OpenYiff/Kemono but this is for running a kemono.party service. Any ideas? I got used to using yiff.party as a backup/3rd party ripper to get HD content to hydrus, then deduplicate the shit booru quality versions.
9 posts omitted.
>>15022 It was broken for a while but it seems to be working now. >>15020 The site has changed the URL schemes for posts since that was put on Cuddlebear's repo. I would wait until the site is more stable and/or the API is finished before trying to find/make a downloader.
>>15024 I see, that makes sense. Thanks.
I made that WIP kemono parser, its specifically for kemono.party/files and was made when the actual kemono frontend was non-functional. It was meant to do the bare minimum of scraping their bulk file CDN but I don't even think they use the same URLs for file downloads anymore.

(14.63 KB 480x360 YnU_j_ZA-tc.jpg)

Version 424 Anonymous 01/07/2021 (Thu) 01:59:59 Id: 0070e5 No. 15072 [Reply]
https://www.youtube.com/watch?v=YnU_j_ZA-tc windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Linux.-.Executable.tar.gz I had a good week. There are some quality of life improvements and faster tag search across the board. The update will take some time this week to update a cache. If you do not sync to the PTR, it will be just a few seconds. If you sync to the PTR, expect about 5-15 minutes. faster tag search In the second half of 2020, I tried several times to tune the database for different sorts of wildcard tag search, which is used in all autocomplete lookups and many file searches. I was sometimes able to get small clients always running well, or complicated large systems running, well, but I failed to get it good for all situations with code aloneโ€“the structure of the database tag lookup cache made the tuning difficult.

Message too long. Click here to view full text.

4 posts and 1 image omitted.
>>15090 Yes I agree, I will have some sort of client-to-client transfer as part of this, so users who have been using multiple clients will be able to merge neatly and preserve time and archive metadata. Around that time I'll likely have to introduce rating export/import too, and perhaps URLs as well. The good news is now we have the Client API, this should be neatly possible as a direct client-to-client connection.
>>15092 Thank you.
I had a good week. I was not able to fit much interesting fun stuff in, but I fixed the slow tag autocomplete search in the manage tags dialog, sped up tag processing, fixed some 'ghost' tag bugs, and reduced some wasted CPU in the network engine. Overall, the client should be a bit neater and faster in 425. The release should be as normal tomorrow.

(94.10 KB 1024x768 1601788051705.jpg)

/big data general/ Anonymous 10/05/2020 (Mon) 05:17:17 Id: ecf9e3 No. 14802 [Reply]
or, "i've got xxxxxxx thousand images in my inbox, what the fuck do i do?" i'm currently having this problem myself, so i thought it would be nice to have a thread to collect information about various ways to automate sorting/tagging/etc, such as: * image deduplication scripts: >https://github.com/knjcode/imgdupes i'm presently also writing my own as well which should work somewhat better for large datasets, i'll post it here when it's usable * datasets, useful for running against aforementioned dedupe scripts >danbooru2019, contains all danbooru pictures + metadata up to early 2019: >https://www.gwern.net/Danbooru2019 i remember there being large dumps of other booru metadata on here years ago, but i can't find them anymore * AI/neural network software for automated tagging, classifiers and etc: https://github.com/KichangKim/DeepDanbooru https://github.com/imamar94/ramrem-classifier things i couldn't find but would find extremely useful: >AI anime/photograph classifier

Message too long. Click here to view full text.

4 posts and 1 image omitted.
>>15084 Is that not what it does? Unless you mean applying a non-system tag like "video with audio" or something, it does do that automatically.
>>15099 Just tried adding a new url with a bunch of videos to the watch, some have audio some don't. None of them have been autotagged with any tags, except for the few that have a hash matched. So it does not seem as if it is auto-applying tags for stuff that are videos, and stuff that has audio.
>>15101 do a search for "system:has audio" you absolute retard

(100.78 KB 1624x1081 1327614072601.png)

Suggestions Anonymous 03/18/2015 (Wed) 23:36:12 Id: 68f861 No. 471 [Reply] [Last]

Drag and drop windows with tag rules. Show two windows side by side and one window can be programmed with the rule "ADD tag foo" and the other one has the rule "REMOVE tag foo, ADD tag bar" and you can drag and drop files to them.

Deriving tags from regex of other tags/namespace tags. A file has the tag "filename:big_ugly_name" and we could regex that namespace for another tag.

Tag sets with hotkeys: save a set of tags under a hotkey so it's quick to add them to a file while filtering

Opaque window behind tag list in the corner so it doesn't get hidden by picture background

Option to default certain mime types to be excluded from slideshow and only open externally, will help with videos with odd codecs that don't preview in the slideshow correctly

Option to specify hamming distance in "find similar images", you can't change the option once it's in the filter window and you have to enter the hash manually in the "system:similar to" option
743 posts and 1 image omitted.
>>15091 If you set tag import options in the watcher page it is applied to the threads that are queried while those are set, so you could change it to have your tags for that thread then change it again for another thread. You can change the amount of recent tags in options, options>tag suggestions>recent. Shift clicking to select a thumbnail range should work already.
>>15094 >If you set tag import options in the watcher page it is applied to the threads that are queried while those are set It would be very nice to have a prompt or option area to add a default tag if the user wishes. Then it just resets after the page is submitted. The user could also right click the watch and edit the tags and it would easily update them for all the previously grabbed files in that watch and any newly tagged files. The way you explain is a bit inefficient imo. >You can change the amount of recent tags in options Ah, a bit hidden, thank you. >Shift clicking to select a thumbnail range I was behind like 2 Hydrus updates, but just updated and that works now. Maybe it was a bug then. Works now though. Thanks.
Does Hydrus have any support for multiple audio/video streams in one file? Say I have 4 videos with the same audio, just slightly different video (say different levels of clothing). You can use ffmpeg or what have you to combine them into one file with multiple video streams and one audio stream, which mpv can play (at least externally, cycling with the _ keybind). You could do similar things for multiple videos which only differ in audio, say language (keybind #). You could even have both multiple video and audio streams in one file, if you really wanted. I could have each video+audio stored separately in Hydrus, but to me they're the same video, just different variants. Being able to combine them would help reduce clutter in the search and lets you be sure just from the thumbnail that the variant you want is available. I think it would be cool if Hydrus could switch video/audio streams without having to launch in an external program. Don't know the technical details on whether a keybind or right click menu or both would work, though.

Organizing 2D and 3D? Anonymous 01/07/2021 (Thu) 16:38:38 Id: 4280d7 No. 15079 [Reply]
Do you only save 2D to Hydrus, or both 2D and 3D? How do you go about organizing different file types? For example, 2D Photos, 3D Photos, Books/Doujins, etc. I've stayed away from archiving any types of Books/Doujins into Hydrus, and keeping that into Calibre. As for videos, anything full length (stuff you wouldn't find on Boorus) I just run it through Jellyfin. So I guess personally my biggest thing is 2D and 3D photos/short videos/animations. Right now, I just have them seperated with a tag. Filetype:2D, Filetype:3D. I have my import folders seperate (3D folder and 2D folder) and they import with that tag automatically if stuff goes in there. Is there a more organized way? I can't say I'm entirely a fan of having both 3D and 2D in the same database. 99.9% of my 3D stuff I don't even tag, it's only there so I can view it. So it feels useless to be in hydrus. Opinions?
>>15079 I'm fairly new to Hydrus myself. If your using Hydrus to view your 3D but not tagging it, so what, no big deal if you still like the software and UI for simply browsing images. On the upside, all your images are stored in one repo, not all spread out everywhere, so you have one place to manage them all. I would say that's worth it right there. My only recommendation would be to maybe find some general tags you could use for your 3D stuff, to be able to sort that easier. As your collection grows, seems like it would be easier to search and sort in the future. Go with what workflow seems to work for you.
I have an extremely small amount of 3D in hydrus, I tag it with "series:real life". I have to manually tag it, but since I can count the amount of 3D I have on one hand it's not too much of a pain. I manage my anime, manga, and books outside of Hydrus, but I keep doujins in it. I tag doujins differently from images, kind of like how exhentai has different tags than boorus do. One of the best parts of Hydrus is the archive/delete filter. I regularly go through my archive and trim the fat, there's a ton of images that look good the first time but that you realize you would never actually jerk it to. If you have tons of files that cannot be automatically tagged, I recommend doing so to make it easier on you.

(20.80 KB 480x360 SvsHVu3xt6A.jpg)

Version 423 Anonymous 12/23/2020 (Wed) 23:40:54 Id: 4a7246 No. 15033 [Reply]
https://www.youtube.com/watch?v=SvsHVu3xt6A windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Linux.-.Executable.tar.gz macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v423/Hydrus.Network.423.-.Linux.-.Executable.tar.gz ๐•ธ๐–Š๐–—๐–—๐–ž ๐•ฎ๐–๐–—๐–Ž๐–˜๐–™๐–’๐–†๐–˜! I had a good week making some small fixes and improvements to finish up the year. This is the last release of the year. There is a large poll on what 'big thing' to work on next: poll Here is the poll on what large work to go for next:

Message too long. Click here to view full text.

15 posts and 1 image omitted.
I had a good work week. I did a variety of small fixes and quality of life improvements, and I finished redesigning the part of the database that does wildcard tag searches. Autocomplete lookups and file searches that rely on complicated tags are running faster across the board, and the old design's sudden lag spikes (e.g. with namespace:*anything* searches) are entirely eliminated. The release should be as normal tomorrow. There will be some database work on update. I will have a better idea tomorrow, but I estimate users who sync to the PTR with an SSD can expect it to take 5-15 minutes.
>>15059 I did already try that, however it seems to only accept individual galleries (and at 'browsing' quality, at that). If I had to seek out each gallery and check for new ones myself, it's not really much of an upgrade. I expect to get something downloading via the archiver would be a pain if for no other reason that there's multiple pages and you would need to check that the user has enough gp/currency (people might also get upset if gp gets wasted). Not to mention the archiver downloader will be in a compressed filed that will need to be extracted. Still, in principal it should be possible. I and I'm sure other people can help answer questions about exhentai. It would be a big feature IMO.
>>15062 >Seems like JPEGXL with the right parameters is great across the board. Exactly. Matches my own tests on the current Jpeg XL reference. Exciting times. ImageMagick got jxl in 7.0.10-54 >My secret dream is that the huge 'corpus' of tag metadata we have built up with the PTR so far lets us train ML in the next five years and the scope of the PTR is greatly reduced. Yea, that would be nice. Still might leave artist name, url and so on. Plus just imagine how many more images people might produce via AI crunching / pre-producing images. I wonder if the db of known images (plus known tags - future training data?) will be obsolete all that soon.

(9.56 KB 480x360 AHBJ68icJ_4.jpg)

Version 422 Anonymous 12/16/2020 (Wed) 23:35:56 Id: 88ffc2 No. 15014 [Reply]
https://www.youtube.com/watch?v=AHBJ68icJ_4 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v422/Hydrus.Network.422.-.Linux.-.Executable.tar.gz ๐ŸŽ‰๐ŸŽ‰ It was hydrus's birthday this week! ๐ŸŽ‰๐ŸŽ‰ I had a great week. I mostly fixed bugs and improved quality of life. tags It looks like when I optimised tag autocomplete around v419, I accidentally broke the advanced 'character:*'-style lookups (which you can enable under tags->manage tag display and search. I regret this is not the first time these clever queries have been broken by accident. I have fixed them this week and added several sets of unit tests to ensure I do not repeat this mistake.

Message too long. Click here to view full text.

1 post and 1 image omitted.
>>15014 Two issues I've noticed network-wise so far. Issue 1 Disclaimer: no idea if this is a regression - first time I've tried this on any version. On v421 I'm running into an error when querying for IDs with specific filetypes through the API. I'm trying to get a list of all files of type webm, wav, etc. but I can't figure out the proper way to do that in the "get_files/search_files" endpoint. Stuff like "meta:webm" works in both the client and API but it's missing a lot of files and seems to be manually assigned; on the other hand, using the query "system: filetype is foo" in the client seems to pull directly from the local DB of files and doesn't miss any but when I hit the API with "?tags=%5B%22system%3A+filetype+is+webm%22%5D" I get no results. I'm not sure if this is user error, by design/wontfix, or you just haven't gotten around to handling/exposing it yet. Is there currently a way to filter on the filetype over all local files? If not, could you add that to the suggestions you're polling on? Issue 2 Disclaimer: I have a janky workaround for this I'm working on a browser-based frontend for Hydrus, and one of the problems I've hit is that safari for iOS (and maybe Mac too, I don't have one to test) won't download media like mp4 files because it tries to sample the file with a range download, and if the remote server doesn't support byte ranges then it bails and the file doesn't load. My current workaround is an nginx reverse proxy with the proxy_force_ranges option set, but it's kind of janky and I can't really ask people to run an nginx server to use the webapp if I ever get it to a releasable state. I'm not sure what library you're using to serve the API but if there's an easy way to enable byte ranges that would be great, and if not no worries, it'll be quite a while before it'll matter.
I had a good week making some small fixes and improvements to finish up the year. Autocomplete works a bit faster, and some quality of life is improved. The release should be as normal tomorrow. It will have the 'next big job' poll and be the last release for the year.
>>15016 Hey, unfortunately system predicates do not work on the Client API yet. They aren't text-based in the actual client, so I can't just pull text in the query to accept them (yet). They are top of the list to add for Client API features. Here's the master job for Client API by the way: https://github.com/hydrusnetwork/hydrus/issues/656 Thank you for the note about file ranges. I have not run into this need yet, so I will research how to do it properly. I use twisted as my main server engine, so I know this is possible, I just think I have to link some things together.

[ 123456789101112131415 ]
Forms
Delete
Report