tinkralge

joined 1 year ago
[–] tinkralge@programming.dev 1 points 3 weeks ago (1 children)

You typically want a slightly-more-elaborate approach than just handing the network a hash and then getting a file.

[...]

Blake 3 supports verified streaming as it is built upon merkle trees as you described. So is IPFS. As I mentioned, IPFS hashes are that of the tree, not the file contents themselves, but that doesn't help when you have a SHA256 sum of a file and want to download it. Maybe there are networks that map the SHA256 sum to a blake3 sum, an IPFS CID, or even an HTTP URI, but I don't know of one, hence the question here.

BitTorrent and Hyphanet have mechanisms that do this.

Do you know of a way to exploit that? A library maybe?

[–] tinkralge@programming.dev 2 points 3 weeks ago (2 children)

It's quite simple: I want to retrieveFile(fileHash) where fileHash is the output of md5sum $file or sha256sum $file, or whatever other hashing algorithm exists.

[–] tinkralge@programming.dev 2 points 3 weeks ago (1 children)

I'm not sure what your concern is. I'd basically like to call a function retrieveFile(fileHash) and get bytes back. Or call retrieveFileLocations(fileHash) and get URIs back to where the file can be downloaded. Also, it'll be opensource, so nothing to reverse engineer.

[–] tinkralge@programming.dev 1 points 3 weeks ago

How do I retrieve a file from bittorrent with just its hash? Does WebMirror solve that? I'll have a look at it...

[–] tinkralge@programming.dev 1 points 3 weeks ago

If you sha256sum $file and send that hash to somebody, they can't download the file from IPFS (unless it's <2MB IINM), that's the problem. And it can be any hashing algorithm md5, blake, whatever.

[–] tinkralge@programming.dev 4 points 7 months ago

Sure, why not. I'm not the creator of the room 🙂

[–] tinkralge@programming.dev 2 points 7 months ago (1 children)

I know that @Ategon@programming.dev is there. Maybe he can make you a mod there too? And the channel could be added to the sidebar of this community too.

[–] tinkralge@programming.dev 4 points 1 year ago

This week some more work was done on inheriteRS to support inheriting non-trait implementations of functions.

Basically

use inheriters::specialisations;

specialisations!(
    struct Parent {
        attr1: u8,
    }
    impl Parent {
        fn from_parent(self) -> u8 { 8 }
        fn overridden(self) -> u8 { self.attr1 }
    }
    
    #[inherit(Child)]
    struct Child {
        attr2: u8,
    }
    impl Child {
        fn overridden(self) -> u8 { self.attr2 }
    }
);

results in

struct Parent {
    attr1: u8,
}
impl Parent {
    fn from_parent(self) -> u8 { 8 }
    fn overridden(self) -> u8 { self.attr1 }
}


struct Child {
    attr1: u8, // new
    attr2: u8,
}
impl Child {
    fn from_parent(self) -> u8 { 8 } // new
    fn overridden(self) -> u8 { self.attr2 }
}

There might be some clean-up necessary code-wise and the README has to be expanded. But the project is well on its way to version 1! It's a pity codeberg doesn't have easy CI/CD yet nor domain hosting e.g inheriters.codeberg.io or something. So auto-formatting, testing, auto-tagging, etc. will have to come once I can convince myself to setup a VPS somewhere that hosts all that.

[–] tinkralge@programming.dev 2 points 1 year ago (1 children)

That sounds like fun! Wow. How stable is it at the moment?

[–] tinkralge@programming.dev 4 points 1 year ago

Working on some form of inheritance in rust. It's my first foray into procedural macros and so far it's fun. The idea is quite simple: generate structs with common attributes (and eventually functions) instead writing them yourself.

use inheriters::specialisations;

specialisations!(
    struct Parent {
        attr1: u8,
    }
    #[inherit(Child)]
    struct Child {
        attr2: u8,
    }
);

becomes

struct Parent {
    attr1: u8,
}
struct Child {
    attr1: u8,
    attr2: u8,
}

not

struct Parent {
    attr1: u8,
}
struct Child {
    attr1: u8,
    parent: Parent,
}

The latter leads to indirection which I'm not a fan of.

Last week I squashed one bug on the order of attributes according to inheritance. In the example above attr2 was coming before attr1. A feature is nearly done to exclude the Parent from the output and only output the child. That's useful for parents that just serve as holders for shared attributes.

The goal for v1 was to also support basic inheritance of implementations: Parent has an impl block, then that block is copied for the Child. Not sure yet if I'll implement overrides in v1 or v2. Overrides being if Parent implements do_something() and Child does too, then the implementation of Parent is not copied into the impl block.
That's what I'll try to tackle in the coming weeks.

[–] tinkralge@programming.dev 2 points 1 year ago

Great, thank you! I'll be moving on to mkDerivation soon. Hopefully budding packagers will be less intimidated than I was when I started.

 

It's a WIP, but I hope it's understandable and contains the necessary information to start packaging something using the builtin derivation function.

15
submitted 1 year ago* (last edited 1 year ago) by tinkralge@programming.dev to c/nix@programming.dev
 

Despite the name, the nixlang wiki is a community contributor wiki to document things around nix that the maintainers haven't gotten around to, to short-circuit the lengthy review process for the official documentation, and an attempt to make an entry into the nix ecosystem easier.

It's difficult to write things from a perspective of somebody who doesn't know, despite once being that person!

It would be great to have some feedback on the wiki.

P.S I'm not the maintainer, just a contributor.

[–] tinkralge@programming.dev 3 points 1 year ago (1 children)

Jottacloud is what I want to use. Unlimited storage for ~100€/month

Close behind is 1fichier for 2€/TB/month or 12€/TB/year, but they are in France and "uptobox" (a similar provider) was shutdown by the US on French soil because they allowed providing links to the files.

You can probably find others in the list of storage systems supported by rclone

 

TL;DR No archive format like tar, zip, ... but how would you theoretically represent a symlink in a manner that can be stored on the cloud and retrieved back to the system as a symlink?

Backstory

I heavily use symlinks to organise my media and even wrote an application that helps me do so (it's in Python and being rewritten in Rust). But I also use stuff like home-manager and nix which makes heavy use of symlinks.

My goal is to back up my media and /home to the cloud at regular intervals. There are services that cost just about 60-100€ yearly for limitless storage in the cloud. So having part of my library purely in the cloud and using terrabytes of space would cost less than a single 15TB HDD (500+€). To have a local backup, I'd even need a least a second one, which would put me at >1000€ - the equivalent of at least 10 years of cloud storage.

Options explored

rclone

It is pretty sweet as it supports mounting a cloud drive as a folder and has transparent encryption! However there are multiple open issues on uploading symlinks and I don't know Go. I wouldn't mind trying to learn it if I had an idea how to upload a symlink without following it (following symlinks breaks them).

git-annex etc.

git-annex and using a bare git repo with a remote worktree is great, but I don't need to make diffs of stuff and follow how things moved around, etc. I just need to replace backups with a view of what's there. Plus, storing all that history will probably take enormous amounts of space which is wasteful.

Ideas

store a blob of stat() call for every file

I'm not sure about this. The stat struct does contain information about the filetype (directory, hard link, symlink, ...), but my knowledge of linux internals is limited and maybe that's too complicated for this usecase.

a db of links

Instead of storing the links themselves, I store a DB (sqlite? CSV?) of links, upload that DB and use the DB to restore links after pull it back down. 🤔 Actually this might be the simplest thing to do, but maybe y'all have better ideas.

view more: next ›