[repoman] first db and tool syntax work

[Dan McGee]

On 7/7/07, Paul Mattal <paul at mattal.com> wrote:
> So here's a first shot at the DB model:
>
>    https://subzero.elys.com/svn/repoman/trunk/repoman/repos/models.py

I don't do a whole lot of work with databases, so looks good to me. :)

If we are going to use sha1sums (which seems fine to me for now until
we get into any GPG stuff or something), then you can cut that
signature field down to size 40 (or 41 if you need the null
character).

> And here's my first thoughts on what the syntax of the tool would look like.
>
>    https://subzero.elys.com/svn/repoman/trunk/doc/REPOMAN.txt
>
> Of these, the second seems more tentative to me right now. There could
> be a neater way to do this. I was thinking about if we should make the
> commandline options as pacman-like as possible or not. I have spec'd
> this out doing more word operation options, like you encounter in SCMs
> commandline tools.

I like SCM-like operations. They are rarely ambiguous, unlike the -c
flag in pacman which has two drastically different meanings.

> Happy to hear thoughts on these!
>
> Some things I thought about in process:
>
> 1) The repoman CLI can act as upload client and as management client. It
> would be nice to just have the client authenticate and talk to the DB
> directly for making updates. I think most DBs like postgres and mysql
> offer a way to do encrypted connections directly to them, and hopefully
> these mechanisms of connection are supported through the Django and
> Python DB abstraction layers.

If we can do something like this, or at the very least have a minimal
server program that doesn't always need to be running and can be
communicated with through some secure channel, that would be good.

> 2) The commit step gets separated from the upload step so you can do a
> bunch of work and then upload all the new packages at once via the most
> efficient transport. Thanks to package signatures, when you choose to
> put the same binary package into another repo, the server can tell it
> already has that package and doesn't require you to re-upload it!

I like separating these two steps, however some issues come in to
play. We need this system to work for two different ways of operation:
automated and supervised builds. As long as we can find a way to
accommodate signatures from both of these types of methods, we should
be fine.

What about some kind of field in the BinaryPackage table that
indicates how the package was/is built? This could perhaps signal the
server to accept a commit without a sha1sum at commit time. The
autobuilder would then fill this data in when it finished the build.
This field could also be used to enforce testing of automatically
built packages, perhaps.

> 3) There needs to be a custom server component of some sort anyway in
> order to handle updating the binary repositories themselves on disk and
> rebuilding the db files. So more than ever, I'm resolved to write that
> server component properly with SSL connection to the client and have it
> handle the file transfer step, too. I'll probably start with the
> codebase from the AUR tupkgs server.

+1, and probably not bad to at least glance at the old codebase. WRT
reuse- unless it is exactly what you want, don't try to meld it into
shape- just rewrite it. (/me kicks himself for not doing this with
libalpm)

> 4) Repos can depend on other repos. This means that it can only have
> dependencies in the same repo or in repos in its dependency tree. So
> extra depends on current, and this keeps you from putting packages into
> current that depend on packages in extra.

How do you plan on enforcing this? And what if makedepends are allowed
to break the rule, but not depends?

-Dan