N0x0n

joined 1 year ago
[–] N0x0n@lemmy.ml 7 points 9 hours ago* (last edited 9 hours ago) (2 children)

Back in the day, that's what I did ALOT on Windows. Specially because of piracy and my younger me having no idea what he was doing XD !

Still happens on Linux with EndeavourOS but not for the same reasons ! There are millions times more ways to break stuff on Linux but I always learn Something new during the process.

Story time:

Learned the other day that some config files are loaded in a specific order and depending what display manager is installed. That was kinda eye opening to understand cause my system didn't load .profile when .bash_profile was present and I didn't understood why ! Thanks Archwiki !

[–] N0x0n@lemmy.ml 16 points 17 hours ago* (last edited 9 hours ago) (1 children)

For those not aware, nepenthes is an example for the above mentioned approach !

[–] N0x0n@lemmy.ml 3 points 17 hours ago (1 children)

I have also observed this behavior, but only when the file is new and the only seeder is the uploader.

Hower I'm kinda curious to test it out. Any idea if there's some way to script qbittorrent to somehow mimic this behavior?

Download until 99,0% completed with 100% of download speed, then reduce to 0.1% of my download speed until finished?

[–] N0x0n@lemmy.ml 8 points 3 days ago (3 children)

It's not because you can't check on Windows, that it doesn't exist ! I'm sure there are a lot of different boot issue logs in Windows, they are just hidden behind a "beautiful" Welcome page.

[–] N0x0n@lemmy.ml 8 points 3 days ago

Or the EU might revisit that old SuSe/Mandrake strategy.

They actually are ! I have seen a few posts talking about it. Not sure about SuSe/Mandrake, but they are talking to implement Linux or try to somehow get away from Microsoft.

[–] N0x0n@lemmy.ml 1 points 4 days ago (1 children)

Naah ! Most flac I have are in the 50/80 range but when you have a huge library it doesn't make sense to archive in flac IMO.

It makes sense if you are music producer, but my whole library would need a huge SSD cluster :/

[–] N0x0n@lemmy.ml 1 points 4 days ago (1 children)

Yeaaah ! That's even better ! However, I'm listening to some obscure Electronic music which often doesn't come in flac format :(

[–] N0x0n@lemmy.ml 2 points 4 days ago* (last edited 4 days ago)

They are probably scared of Ninjas? 😁

[–] N0x0n@lemmy.ml 20 points 4 days ago* (last edited 4 days ago) (6 children)

Maybe enjoy crispy 128 .opus? Yeah flacs are on another level... But 100MB/file just doesn't justify the negligeable quality increase you won't even notice anymore in your late 40s !

However, saving TB of disk space does !

[–] N0x0n@lemmy.ml 2 points 5 days ago

Ohhh yeah ! Haven't though that far, you're right. That's probably the reason why it takes so much time.

[–] N0x0n@lemmy.ml 7 points 5 days ago (2 children)

He wants the German company to pay 17,000 euros ($18,400)

That's peanuts for them... And it took more than 12 years to even get a hearing? Disgusting...

[–] N0x0n@lemmy.ml 9 points 6 days ago* (last edited 6 days ago)

Yeah Lemmy is very Linux oriented :). Though I do agree Linux is harder to get into but that's not because GNU/Linux is difficult, it's because we were so much accustomed to how Windows work !

I don't really remember because it has been ages, but I'm sure my first steps with Windows 98 where awful and not easy and intuitive at all !

Linux has a lot of issues so does Windows (and MacOS) and learning a whole new paradigm of a working system takes time and some investement, but after you get the basic gists, Linux gives you total control of everything ! (Like to see hidden directories is like Windows, just a check box to show hidden files :) don't need a CLI for that)

You rarely need the use of the Terminal when on easier distributions like PopOS or Ubuntu (while I wouldn't suggest the later), However if you tried more difficult distribution first (like arch and derivatives), CLI is mandatory !

Not saying YOU should switch, do whatever makes you happy, more productive and vibes with your current needs/feeling...

However, If you feel the need to switch the Linux community will always welcome you :) Keep in mind, there are good people and a**holes everywhere !

Glad you solved your issue with massgrave, which is a great tool :) ! Have fun with your system !

Edit: Sorry for the off topic XD discussion got a bit heated up haha ! I'm kinda surprised it didn't got deleted !

 

YouTube link: https://youtu.be/wVyu7NB7W6Y

Invidious link: https://inv.nadeko.net/watch?v=wVyu7NB7W6Y

Sorry for the formatting... Tried to remove the URL for better readability, but there seems some kind of bug.


TLDW

  • hack phones remotely just knowing it's phone number
  • Intercept 2FA sms
  • Intercept phone calls
  • Reroute phone calls
  • Geolocation of a target

I dunno if it has already been posted/discussed here but this kinda blew my mind ! Sorry there's a lot of clickbait but the general subject is interesting...

I never heard of SS7 and have actually no idea how the whole phone system communication works but that's kinda scary...

Yes we are probably not the first target with this "hack" nor is it as easy as exposed in this video and nor do we have 14k $ to spend on this, but that's not out of reach for some people. I mean it's not as expensive as Pegasus and people with the mean and some good stable income can probably misuse this system for targeting specific vulnerable people (example in the video).

 

Hello everyone :).

Trying to keep it short, cause after 2 days of troubleshooting I'm a bit tired and really confused on what happened here... Maybe it's my lack of understanding about Legacy BIOS/UEFI/EFI/Bootloader/GRUB... But that was really an odd "issue" that resolved by itself?

Intro


My 15 years old laptop, an Asus N76V, is still going strong, though It's purpose is not the same as a few years back. It's working great as a mini-server to host docker containers, DNS server, firewall, wireguard tunnel...

Space left on my volume group (LVM) was getting tight so I decided to install a new Samsung 1TB SSD into it's second slot. As easy as it is, I though this would not take more than 30 minutes...

Old SSD just vanished from boot option in bios.


After booting into BIOS to see if the new SSD is recognized, everything seems okay, however my primary SSD containing the Bootloader and system just vanished as boot option?? I was not to angry about it because if something strange had happened and everything got wiped for whatever reason, I still have my daily backups, so the troubleshooting begins...

Things I have tried to bring my primary SSD back as boot option in BIOS


Though my secondary and new SSD shows as boot option, it's empty and has no Bootloader or system installed.

  1. Change a lot of things in the BIOS

From AHCI to password protection and disable anti-theft, secure boot... I think I changed every option possible (related to SSD) in the BIOS and reverted back to default without my primary SSD showing again as boot option.

  1. Clear CMOS

Opening the case and taking out the little battery to clear the CMOS had not effect at all either.

  1. boot into Debian rescue mode, boot-repair, chroot session

From a Live session USB tried a few things in rescue mode and even in the Live session used the boot-repair tool and tried to manually fix the EFI/Bootloader in a chrooted environment, because Yes even if I couldn't boot into my system my data was still there an safe !!

  1. Disconnect new SSD, swap bay position.

Even when I removed the new SSD from it's SATA connection or swaped SATA bay connections, the primary SSD didn't showed back into the BIOS....

Guess I have to reinstall Debian on my new SSD?


So I gave up on trying to fix the Bootloader and primary SSD and just went for a fresh Debian install on my new SSD.

Nothing uncommon during the installation process except that during the partition all my volume groups and logical volumes from my primary SSD are visible, so I left them alone and created a new VG and LVs. I do as usual a LVM manual partioning with EXT4 filesystem with separate /boot partition, however I forgot to set the ESP partition (was getting a bit late and got a bit on my nerves...). Installation went without issues.

And then It came back....


So booting into my BIOS to see if my new SSD boot position is okay... Ohhhhh and what a surprise to see my primary SSD back as boot choice... however my new SSD isn't there anymore (expected as I forgot to set an ESP partition... And Bootloader is from my primary SSD).

Booting into my system I'm greeted by the Bootloader? GRUB? With 2 choices

  • My old linux OS
  • My new linux OS

Some kind of relief and happy moment after 2 days...

Kinda curious what happens If set the /boot/EFI partition on secondary SSD


I got kinda confused on what happened here so to further confuse myself I reformatted and reinstalled Debian on the secondary disk with the correct /boot/EFI partition.

And I got even more confused... The boot priority in BIOS only shows the secondary SSD as boot option...!?

Questions


What happend here?

  1. Why did my primary SSD (which had a proper Bootloader and clean system) disappeared as boot option in the BIOS as soon as I installed a new blank SSD and didn't came back even when the new one was unplugged from SATA?

  2. Why does my BIOS only shows 1 disk as possible boot device when both have proper Bootloader and system?

  3. Does 1 external Bootloader suffice to make both system work?!

Not sure about the last question... so maybe I'm looking more or less to sharpen my understanding of a Bootloader/EFI/UEFI/GRUB... And any hint to a good resource, book, eBook to get a better understanding is really appreciated :)


I know those are a lot of words (and I said I will keep it short...) but I think the context is important here, and I'm not able to express correctly my issue/thoughts without it.

Thanks in advance for those who beared with me and read the whole text and can hint me to the correct direction to a better understanding on what happened.

1
submitted 1 month ago* (last edited 1 month ago) by N0x0n@lemmy.ml to c/linux@lemmy.ml
 

Hi everyone !

I'm in need for some assistance for string manipulation with sed and regex. I tried a whole day to trial & error and look around the web to find a solution however it's way over my capabilities and maybe here are some sed/regex gurus who are willing to give me a helping hand !

With everything I gathered around the web, It seems it's rather a complicated regex and sed substitution, here we go !

What Am I trying to achieve?

I have a lot of markdown guides I want to host on a self-hosted forgejo based git markdown. However the classic markdown links are not the same as one github/forgejo...

Convert the following string:

[Some text](#Header%20Linking%20MARKDOWN.md)

Into

[Some text](#header-linking-markdown.md)

As you can see those are the following requirement:

  • Pattern: [Some text](#link%20to%20header.md)
  • Only edit what's between parentheses
  • Replace space (%20) with -
  • Everything as lowercase
  • Links are sometimes in nested parentheses
    • e.g. (look here [Some text](#link%20to%20header.md))
  • Do not change a line that begins with https (external links)

While everything is probably a bit complex as a whole the trickiest part is probably the nested parentheses :/

What I tried

The furthest I got was the following:

sed -Ei 's|\(([^\)]+)\)|\L&|g' test3.md #make everything between parentheses lowercase

sed -i '/https/ ! s/%20/-/g' test3.md #change every %20 occurrence to -

These sed/regx substitution are what I put together while roaming the web, but it has a lot a flaws and doesn't work with nested parentheses. Also this would change every %20 occurrence in the file.

The closest solution I found on stackoverflow looks similar but wasn't able to fit to my needs. Actually my lack of regex/sed understanding makes it impossible to adapt to my requirements.


I would appreciate any help even if a change of tool is needed, however I'm more into a learning processes, so a script or CLI alternative is very appreciated :) actually any help is appreciated :D !

Thanks in advance.

 

Hello :)

There isn't any community about note taking where I could post my question and no this is not a "What's the best note taking app" question...

I'm getting tired of maintaining my Obsidian vaults... Somehow I'm fighting to get it right and obsidian seems to fight back. I've got 4 vaults of the same subject and I always end to make a mess out of it and make a fresh one... Also my notes a scattered in all direction and the more my knowledge base grows the less I seem to be able to find something...

This is probably a me problem rather than Obsidian issue. The way I'm taking notes are not compatible with Obsidian. IMO Obsidian's defaults configuration are bad and visually not appealing. Sure customization in Obsidian is "endless" but digging in the HTML code to change the style or adding plugins to somehow get something visually appealing seems more like a chore than actually taking notes.

Here I'm again roaming the web for a Note taking app the could fit my needs and after trying a lot of different apps (please don't suggest the already well known apps... I have probably already tried it...) I couldn't find something that fits my workflow.

The only one that looked great and simple was osmosnote but it isn't maintained anymore. There's also dendron but it's in maintenance mode. So there goes the only ones that looked promising from my perspective.


After giving it more thoughts, I was looking for something that could:

  • Keep my scripts updated
  • Simple markdown text
  • No database
  • Local first
  • Open source
  • If webapp self-hostable
  • Back-linking
  • Keep track of changes

Except for back-linking, a self-hosted Forgejo with git seems to fit all my needs, however I'm not sure if this is the right tool and I'm scared that in the long run I will mess it up the same way I did with Obsidian.

Does anyone here has some experience and is taking notes that way? I'm really curious on your experience and maybe your thoughts if it's feasible ? Practical ?

Please don't suggest Org.mode or Emacs ! They look very cool and very promising but they are WAY to much overkill ! And they also implement a totally new way of taking notes... Relearning on how to take notes will probably give me the last hit on abandoning to document anything !

Thank you for any helpful input !

 

cross-posted from: https://lemmy.ml/post/23615167

For better visibility I cross-post my question in this community.

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I'm self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

  • Reboot the phone => Same behavior
  • Reboot the piped instance => Same behavior
  • Uninstall RiMusic/New docker piped instance => Same behavior
  • Flush everything from cache/playlist/configuration/data... => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I'm the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

  • Create an account and connect to a public piped instance
  • Create playlist/add some songs
  • Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance...

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And .... RiMusic playlist connected to my piped instance works without the ratelimite lines... Wait what ??

What just happend ? I have absolutely no idea... I don't even know if the mentioned labels did anything... But everything works... No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request... It just vanished as it never existed in the first place.

I'm totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that's the correct term :/). Here Am I with no idea what actually happend...

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !


Sorry for the long write up I hope It's readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I'm open to every good read !

 

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I'm self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

  • Reboot the phone => Same behavior
  • Reboot the piped instance => Same behavior
  • Uninstall RiMusic/New docker piped instance => Same behavior
  • Flush everything from cache/playlist/configuration/data... => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I'm the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

  • Create an account and connect to a public piped instance
  • Create playlist/add some songs
  • Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance...

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And .... RiMusic playlist connected to my piped instance works without the ratelimite lines... Wait what ??

What just happend ? I have absolutely no idea... I don't even know if the mentioned labels did anything... But everything works... No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request... It just vanished as it never existed in the first place.

I'm totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that's the correct term :/). Here Am I with no idea what actually happend...

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !


Sorry for the long write up I hope It's readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I'm open to every good read !

9
submitted 4 months ago* (last edited 4 months ago) by N0x0n@lemmy.ml to c/cat@lemmy.world
 

Oupsii !

1
submitted 5 months ago* (last edited 5 months ago) by N0x0n@lemmy.ml to c/linux@lemmy.ml
 

TIL something new... My hate for MacOS took over common logic. 2.8GB, 3 seconds file transfer on USB was to beautiful to be true. After some further investigation and hints from @JonnyRobbie@lemmy.world @nanook@friendica.eskimo.com I learned that Linux writes to cache before writing it to the device, to see whats happening in the background: sync & watch -n 1 grep -e Dirty: /proc/meminfo.

Still, the transfer speed on Linux was slightly faster than on MacOS. My rant was unjustified, It just my fault for being clueless on some more advanced Linux stuff. But I learned something new today, so this post was actually helpful !

Howerver, I still hate MacOS and will probably give Asahi remix a try.

Thanks to everyone !


Hey guys ! I'm getting tired/bored of MacOS' shenanigans... Yesterday was the last drop that make me think of trying an alternative.

While trying to upload a 2.8 GB file over to an USB-C stick it took like 8 minutes? Okay that's "good" enough if you only do it from time to time... But 25 files takes literally 1h30min... Are we in 2001?

I mean the exact same 2.8GB file, with the exact same USB-C stick took FU***** 3 seconds on Linux !!

Ohh and don't think I didn't tried to "fix" the issue, after a long search on the web I came across a lot of people having similar issues that aren't fixed since 2 major updates? With a total radio silence from the shiny poisonous Apple...

Among other things I tried:

  • Disable Spotlight indexing sudo mdutil -a -i off
  • Reformat the USB stick from Mac
  • All available filesystem FAT32, exFAT...(yes even MacOS native APFS)
  • Another USB stick
  • ....

Enough is enough. I was willing to learn their way of thinking for my personal experience and somehow always got my way around to reproduce what I learned on Linux to Mac. But now that there is an alternative OS, I think I'm ready to get back home.

So does anyone here already gave Asahi Remix a try? If so what was your experience with it?

I read their FAQ and most of their documentation and it seems good enough for daily drive (except for some quirks here and there) but I wanted to hear from people who already made the jump and how was their personal feeling.


PS: I got that MacOS for my birthday from a family member with good intentions. That wasn't a personal choice. While I'm more than happy and thankful for the gift, I totally hate it more and more... Especially because MOST of my self-hosted services, applications, scripts, are open source.

 

Hi everyone !

Intro

Was a long ride since 3 years ago I started my first docker container. Learned a lot from how to build my custom image with a Dockerfile, loading my own configurations files into the container, getting along with docker-compose, traefik and YAML syntax... and and and !

However while tinkering with vaultwarden's config and changing to postgresSQL there's something that's really bugging me...

Questions


  • How do you/devs choose which database to use for your/their application? Are there any specific things to take into account before choosing one over another?

  • Does consistency in database containers makes sense? I mean, changing all my containers to ONLY postgres (or mariaDB whatever)?

  • Does it make sense to update the database image regularly? Or is the application bound to a specific version and will break after any update?

  • Can I switch between one over another even if you/devs choose to use e.g. MariaDB ? Or is it baked/hardcoded into the application image and switching to another database requires extra programming skills?

Maybe not directly related to databases but that one is also bugging me for some time now:

  • What's redis role into all of this? I can't the hell of me understand what is does and how it's linked between the application and database. I know it's supposed to give faster access to resources, but If I remember correctly, while playing around with Nextcloud, the redis container logs were dead silent, It seemed very "useless" or not active from my perspective. I'm always wondering "Humm redis... what are you doing here?".

Thanks :)

 

Edit

After a long process of roaming the web, re-runs and troubleshoot the script with this wonderful community, the script is functional and does what it's intended to do. The script itself is probably even further improvable in terms of efficiency/logic, but I lack the necessary skills/knowledge to do so, feel free to copy, edit or even propose a more efficient way of doing the same thing.

I'm greatly thankful to @AernaLingus@hexbear.net, @GenderNeutralBro@lemmy.sdf.org, @hydroptic@sopuli.xyz and Phil Harvey (exiftool) for their help, time and all the great idea's (and spoon-feeding me with simple and comprehensive examples ! )

How to use

Prerequisites:

  • parallel package installed on your distribution

Copy/past the below script in a file and make it executable. Change the start_range/end_range to your needs and install the parallel package depending on your OS and run the following command:

time find /path/to/your/image/directory/ -type f | parallel ./script-name.sh

This will order only the pictures from your specified time range into the following structure YEAR/MONTH in your current directory from 5 different time tag/timestamps (DateTimeOriginal, CreateDate, FileModifyDate, ModifyDate, DateAcquired).

You may want to swap ModifyDate and FileModifyDate in the script, because ModifyDate is more accurate in a sense that FileModifyDate is easily changeable (as soon as you make some modification to the pictures, this will change to your current date). I needed that order for my specific use case.

From: '-directory<$DateAcquired/' '-directory<$ModifyDate/' '-directory<$FileModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/'

To: '-directory<$DateAcquired/' '-directory<$FileModifyDate/' '-directory<$ModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/'

As per exfitool's documentation:

ExifTool evaluates the command-line arguments left to right, and latter assignments to the same tag override earlier ones.

#!/bin/bash

if [ $# -eq 0 ]; then
    echo "Usage: $0 <filename>"
    exit 1
fi

# Concatenate all arguments into one string for the filename, so calling "./script.sh /path/with spaces.jpg" should work without quoting
filename="$*"

start_range=20170101
end_range=20201230

FIRST_DATE=$(exiftool -m -d '%Y%m%d' -T -DateTimeOriginal -CreateDate -FileModifyDate -DateAcquired -ModifyDate "$filename" | tr -d '-' | awk '{print $1}')

if [[ "$FIRST_DATE" != '' ]] && [[ "$FIRST_DATE" -gt $start_range ]] && [[ "$FIRST_DATE" -lt $end_range ]]; then
        exiftool -api QuickTimeUTC -d %Y/%B '-directory<$DateAcquired/' '-directory<$ModifyDate/' '-directory<$FileModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/' '-FileName=%f%-c.%e' "$filename"

else
        echo "Not in the specified time range"

fi



Hi everyone !

Please no bash-shaming, I did my outmost best to somehow put everything together and make it somehow work without any prior bash programming knowledge. It took me a lot of effort and time.

While I'm pretty happy with the result, I find the execution time very slow: 16min for 2288 files.

On a big folder with approximately 50,062 files, this would take over 6 hours !!!

If someone could have a look and give me some easy to understand hints, I would greatly appreciate it.

What Am I trying to achieve ?

Create a bash script that use exiftool to stripe the date from images in a readable format (20240101) and compare it with an end_range to order only images from that specific date range (ex: 2020-01-01 -> 2020-12-30).

Also, some images lost some EXIF data, so I have to loop through specific time fields:

  • DateTimeOriginal
  • CreateDate
  • FileModifyDate
  • DateAcquired

The script in question

#!/bin/bash

shopt -s globstar

folder_name=/home/user/Pictures
start_range=20170101
end_range=20180130


for filename in $folder_name/**/*; do

	if [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateTimeOriginal "$filename") =~ ^[0-9]+$ ]]; then
		DateTimeOriginal=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateTimeOriginal "$filename")
	        if  [ "$DateTimeOriginal" -gt $start_range ] && [ "$DateTimeOriginal" -lt $end_range ]; then
			/usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateTimeOriginal/' '-FileName=%f%-c.%e' "$filename"
			echo "Found a value"
		echo "Okay its $(tput setab 22)DateTimeOriginal$(tput sgr0)"

		fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -CreateDate "$filename") =~ ^[0-9]+$ ]]; then
                CreateDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -CreateDate "$filename")
                if  [ "$CreateDate" -gt $start_range ] && [ "$CreateDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$CreateDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 27)CreateDate$(tput sgr0)"
                fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -FileModifyDate "$filename") =~ ^[0-9]+$ ]]; then
                FileModifyDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -FileModifyDate "$filename")
                if  [ "$FileModifyDate" -gt $start_range ] && [ "$FileModifyDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$FileModifyDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 202)FileModifyDate$(tput sgr0)"
                fi


        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateAcquired "$filename") =~ ^[0-9]+$ ]]; then
                DateAcquired=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateAcquired "$filename")
                if  [ "$DateAcquired" -gt $start_range ] && [ "$DateAcquired" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateAcquired/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 172)DateAcquired(tput sgr0)"
                fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -ModifyDate "$filename") =~ ^[0-9]+$ ]]; then
                ModifyDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -ModifyDate "$filename")
                if  [ "$ModifyDate" -gt $start_range ] && [ "$ModifyDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$ModifyDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 135)ModifyDate(tput sgr0)"
                fi

        else
                echo "No EXIF field found"

done

Things I have tried

  1. Reducing the number of if calls

But it didn't much improve the execution time (maybe a few ms?). The syntax looks way less readable but what I did, was to add a lot of or ( || ) in the syntax to reduce to a single if call. It's not finished, I just gave it a test drive with 2 EXIF fields (DateTimeOriginal and CreateDate) to see if it could somehow improve time. But meeeh :/.

#!/bin/bash

shopt -s globstar

folder_name=/home/user/Pictures
start_range=20170101
end_range=20201230

for filename in $folder_name/**/*; do

        if [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateTimeOriginal "$filename") =~ ^[0-9]+$ ]] || [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -CreateDate "$filename") =~ ^[0-9]+$ ]]; then
                DateTimeOriginal=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateTimeOriginal "$filename")
		CreateDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -CreateDate "$filename")
                if  [ "$DateTimeOriginal" -gt $start_range ] && [ "$DateTimeOriginal" -lt $end_range ] || [ "$CreateDate" -gt $start_range ] && [ "$CreateDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateTimeOriginal/' '-directory<$CreateDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 22)DateTimeOriginal$(tput sgr0)"

                else
			echo "FINISH YOUR SYNTAX !!"
		fi

	fi
done

  1. Playing around with find

To recursively find my image files in all my folders I first tried the find function, but that gave me a lot of headaches... When my image file name had some spaces in it, it just broke the image path strangely... And all answers I found on the web were gibberish, and I couldn't make it work in my script properly... Lost over 4 yours only on that specific issue !

To overcome the hurdle someone suggest to use shopt -s globstar with for filename in $folder_name/**/* and this works perfectly. But I have no idea If this could be the culprit of slow execution time?

  1. Changing all [ ] into [[ ]]

That also didn't do the trick.

How to Improve the processing time ?

I have no Idea if it's related to my script or the exiftool call that makes the script so slow. This isn't that much of a complicated script, I mean, it's a comparison between 2 integers not a hashing of complex numbers.

I hope someone could guide me in the right direction :)

Thanks !

 

Hi everyone :).

Just getting started with Manjaro as daily drive to get some easier arched based distro. Except for the LVM bug with calamares everything is pretty smooth :).

But at first boot, I saw they have added their personal Manjaro logo on boot and I directly though of the bug exploit logoFAIL I heard a few month ago and It made me curious if this is something that could be exploitable by Manjaro.

Probably not, this would harm their image and hard worked system, but I'm still curious... If someone smarter/more knowledgeable than me could chime in and give some valuable information on this topic regarding Manjaro, I would really appreciate it !

Thank you !

 

cross-posted from: https://lemmy.ml/post/15968883

Hello everyone ! Nobody seems to have an answer on !networking@sh.itjust.works (or maybe they are not interested because it's an enteprise network community?) and !homenetworking@selfhosted.forum seems dead?

Anyway, If anyone could guide me or direct me to the right direction, I would really appreciate it !


TL:DR

What is encapsulated into the frame that makes everyone understand: "OHHH that’s for 10.0.0.8, your docker container on bridge network br-b1de on the veth2b interface !!! "


Hi everyone !

I'm scratching my head in finding an actual answer on how virtual networking in docker actually works (mostly on the packets/frame level) or some good documentation to improve my understanding on how everything fits together.

Because I'm probably lacking the correct network terminology I made a simple network topology of my network. Don't hesitate to correct any network mistake.

In my scenario, my docker container with the virtual interface veth2b22c98 and the following ip (10.0.0.8) connects to bridge network br-b1de95b5ea89. When I curl, from my conntainer, lemmy.ml the packets/frame is send to my enp4s0 and goes through my wireguard tunnel to my VPN provider which sends back the packet/frame/handshake...

I probed every interface with tcpdump (enp4s0, wg0, br-b1,veth2b):

  • enp4s0: Every packet/frame is encapsulated into the wireguard protocol with my physical interface's IP (192.168.1.30) and no DNS is visible on that interface (like expected) and sends it out to my ISP's public IP.

  • wg0: Shows every packet/frame with the actual protocol with my wireguard's interface IP (192.168.2.1) with the destination IP of lemmy.ml (Dst: 54.36.178.108)

  • br-b1: Shows every packet/frame with the actual protocol with my containers IP (10.0.0.8) with the destination IP of lemmy.ml (Dst: 54.36.178.108)


I know there is a mix of 2 different concepts in my scenario (wireguard tunnel and virtual networking) but I really do not understand how the frame gets back to my docker container. When I look at the frames on wg0, there is no mention of either the MacAddress of my container or the actual IP of my container.

How/when/what ? is exactly happening to my frame so that it gets to the correct target between my physical interface, virtual interface, bridge ? I mean with VLAN's there's a VLAN tag on the frame, so you can easily identify with Wireshark where it should go. But here, I cannot find any clue who or what is doing the magic so the frame finds it's way back to my docker container.

What is encapsulated into the frame that makes everyone understand: "OHHH that's for 10.0.0.8, your docker container on bridge network br-b1de on the veth2b interface !!! "


Sorry for my broken English and lack of networking terminology and thank you for those who beared with me and are willing the give me some hints/proper networking lesson.

view more: next ›