* package management
* distributions
* features
* strategy

additional features

this is a list of features a packet manager could have in order to increase usability.

quality assurance

  • packet is checked for manpage per binary
  • packet is checked for standard conform command line options
every gui has to implement --help for example
"-i eth0" always is the network interface, say for nmap/ping/... whatever

this could be used to query a software for implementing certain standards set (for example) by the LSB (linux standard base).

  • postprocessing: after a software is installed a post_check() script could be called which is either contained in the just installed software (warning security issue arising) or which is statically bound in the DPM (decentral packet manager). I could imagine both with dropping privileges for instance the first can be very helpful. Example: You install a software on gentoo but you forgot some useflags, now the check log tells you that not all software was installed. the report is filled with warnings as" component not found".

this framework could be extended to autocheck if certain other parameters are correctly set.

syscall watchdog

when a dependency is not set in an ebuild, the buildstep will fail as the required component is not there. sometimes a different, already installed program, had the same dependency and therefore this problem is not recognized.

solution: the package is generated by a set of tools which might monitor all OPEN system calls (of course all other important calls as well). If any of that calls resolves into a file which is not part of a package the build step MUST fail. this might be possible on unix like operating systems only. also things like RPC (which are not used in the build step to my knowledge) might corrupt this attempt of monitoring files.

generate 'ready to use' virtual machine images

often i got the problem that i just don't want to setup a vm with a certain linux distro. it would be nice if one could 'just' create a virtual machine based from a selection of packets.

references to kernel modules

often i see this problem:

  • user wants to use a device as the usb0 networking device. say the module is not included in the kernel nor is it compiled in. there is usually no way depend on a certain kernel config. portage for instance checks sometimes for kernel configurations and fails to install some software with a warning which is very handy.

we need a way to have hardware dependencies. say i want the usb0 device for usb networking which uses some kernel module. i just force/enable this dependency and it is built automatically.

software security checks

it would be handy to have the exploit db (security database) affecting the system programs by policy as for example:

  • do not start programs which have a critical bug
  • stop execution of software which has a critical bug
  • fix issue automatically
  • detect running programs with have a security issue and stop them from execution

and on periodic updates the status of this policy is enforced.

these kind of features should be managed by the packet manager and the os for example

bug reporting

sometimes a user reports a situation where the dependency graph of the package management system's installation routine fails to work. we need a way to investigate this issue. maybe there is a way to clone the situation:

  • emulate installed software
  • simulate the update process or the installation process (whatever resulted in the issue reported)

now an experienced developer can solve the issue which is probably caused by a set of packages having wrong dependencies.

to sum up: a dev will not have to log into the remote machine. he can copy the 'situation' from there and can solve the problem on his machine. security wise this voids the privacy of the user since a dev can see what software he has installed.

clone or migrate feature

  • this feature can clone the current system to a different one (not the user files)
  • this feature can convert a 32 bit system to a 64 bit system
  • this feature can convert x86 to mips ...
  • helps to migrate to a new machine

can be of help if you want to test something. for instance if the new machine can handle the software and if it performs usable.

authenticity

packets have to be trusted since they run in a security critical environment

p2p

the way packets are retrieved must not matter. current concepts:

  • download source -> build a self made packet
  • download packet
    • via http
    • via p2p
    • via anything else

dependency graph limitation

most distributions don't have integrated the linux kernel in the dependencies.

this is a very painful missing feature to discover

example: (when installing apache 2.2 on gentoo one can see this message)

Messages for package www-servers/apache-2.2.11:
Please note that you need SysV IPC support in your kernel.
Make sure CONFIG_SYSVIPC=y is set.

If you forget to check this you might get very very odd errors or at least missing /dev entries and therefore your software won't find the device.

hardware integration

it would be nice to have not only software components in the dependency graph but also hardware devices and their capabilities. imagine you build an image for a openmoko platform or for a cisco wrt54g. in both cases you need to know what support is there.

this could be used to inform the user about which hardware works with with software (in case of gps once could see that the xyz-gps usb-mouse is supported by gpsd) and order it.

chat

although this sounds odd at first we should be able to have a platform for developer chat and end user chat. users could find solutions this way on how to install some kind of software (normal user hits a problem then -> if a packet has missing dependencies a developer can fix it).

this could also be a forum or a bug tracker as backend but having an interactive and a active platform with full logging would increase the quality of service.

examples would be:

  • irc for general discussion (irc.kde.org #kde-testing and #sunrise)
  • bugtracker (bugs.gentoo.org for portage maintainance)
  • some other developer prefere IM as jabber for instance

my personal request would be a chat per program with full logging. why? often ppl don't know how to operate a software, next they ask in irc and get a good answer. the problem: usually nobody finds the irc log (if logging was enabled at all) afterwards - so the answers get lost. having a chatroom in a special context: "say #firefox" would help to isolate a source to search for answers.

this chats could bring up FAQs in the usual irc way or add bots from which you can get good answers. i've seen many bots who help to answer questions based on keywords or manual input as: !command would for example be parsed by a bot on irc, see (irc.kde.org#qt)

userspace / adminspace - features

definition userspace/adminspace:

userspace in this context means a user installs software in /home/user
adminspace ordinary packet management as we are used to portage/apt-get ...

it is important to note that it is vitally important to have the ultimate packet manager handle both cases with the same tools. an example would be "mac os x" with "gentoo prefix" which is pretty cool. however it requires special care for packet maintainers and therefore it is NOT used in general.

benefits of having usersapce/adminspace packet management

  • adminspace could monitor userspace and require the user to update or force him to update on security issues
  • userspace can freely install programs or probably adapt programs for individual use-cases as mediawiki hacks which is basically a normal mediawiki source base with lots of modifications
  • userspace has the freedom to install packets wanted

backup/restore feature

Most software on my GNU/Linux system is very redundant so why should i backup redundant stuff one might wonder. For faster restore would be one valid point. Debian and Gentoo both have a nice way to get a list of all installed packets which can be used to reinstall all that stuff later one. Still a lot of stuff has to be done manually as backing up kernel configs on Gentoo and grub configs on both. All this should be automated so that the packetmanager also manages the kernel config and the installtion of grub and the grub config handling of course.

A second point why the restore concept of both mentioned distros might be not working is that once you have had outdated packages you might not get old packages for restore since they are gone. Therefore it is vitally important to have old packets around as well.

Conclusion: it is important to have old packets on a internet repository or a p2p like file storage for later restore! It is also vitally important to keep track of all updates in the packet repo like changing versions in a version bump as well as every change which is applied to a packet source (read ebuild).

installed/updated/removed software logging

Gentoo has a very nice system which loggs all installations and updates of packets with an exact time of installation. This helps very much when searching for any fail which comes either with recession as a new version of a packet is installed but not used for quite some time. I had this issue several times and this helped me a lot. One issue in the way gentoo logging is done is the missing information what use flags were used on the installed programs.

wiki principle - with 'unlimited undo'

say you get to know a exploit and some authors say it was found ages ago. you need to have a look at which version the exploit is affecting. say you found a few different versions which were installed on your system in that time period.

the interesting point is now. where these versions installed during that time period and how likely is a compromised system?

|
|-- 1.0.
  \-- 1.0.1
     \-- 1.0.6 (exploit was created)
     \-- 1.0.7 (exploit still worked)
     \-- 1.0.8 (exploit was fixed for any reason, for example someone rewrote the code)
   |-- 2.0 (exploit never worked in the 2.0 branch)
  \-- 2.0.1
     \-- 2.0.2

having a unlimited version history brings this benefits:

  • might be helpful for historical data mining
    • which version was released
    • who did what?
    • a old pattern fixing several packet group dependencies could apply to a 'yet' unsolved problem as well
  • you could check for code quality differences between versions
  • you can monitor activity in projects

if you include also the cvs/svn/git/...(vcs-version control system) trees into the packet management this could open some very new perspectives as:

  • releases tagged by upstream as a release might have shorter bug fixing times
  • even better monitoring about project activity

keep in mind that making a release as .tar.gz (source release) or similar is usually only done to minimize the load on the vcs-server. when we would grab this tagged release once and distribute this automatically a project just needs to make a 'branch' and tag a release which then is automatically processed. (this is a cool feature)

use flag feature

Gentoo has a very unique way treating packages. Most do underestimate the benefit of this system which gives you the freedom of installing a packet with desired dependencies. This system could be modified in a way that use flags are not additive (as they are now) but subtractive meaning: one builds a packet:

  • without the doc use flag: -doc
  • with the qt use flag: +qt

right after the packet was build and installed it gets obvious that the doc useflag would have been very helpful.

in the gentoo way one would have to reinstall the whole packet. which is obviously the way i would have implemented it.

however - an additive approach would have build the +doc anyway but would not install it. this way a packet is split further. this does not work for most useflags since they are designed to minimize functionality and therefore minimize dependency requirements. another benefit is: it helps you to focus on what you need and it will probably reduce the mission critical security issues - when using less code - therefore code that is not needed.

splitting packets, updating using deltas

another key issue is the fact that even different versions of binary packets share a lot of similar bytes. an update to a new version of the software could use the old binary packet and compute a delta (the binary difference of the two packets) and only transfer this. this is no new technology and is already used in the SuSE distribution.

Still all different versions of the packet share common things which i would like to call the base packet and all other things would be installed additionally.

this feature is considered nice to have and will help to minimize disk space usage on the online mirrors for packets. it is basically a time/space trad off but on upgrades a base packet might help to reduce re-transfering redundant files (which are already on the system).

MVC feature

Model View Controller Prinzip [11], fuer updates.

Idee: jemand verwaltet ein pool an rechnern und will auf denen updates machen. Alternativ: jemand will nachsehen welche software auf einem laptop installiert. man koennte die softwareliste (also die db fuer lokal installierte software auch extern hosten).

voteile db extern hosten:

  • update jobs koennen extern berechnet werden (pakete downloads, dependencies)
  • verwaltung fuer rechnerfarm wird einfacher
  • verwaltung von software allg. koennte als baum organisiert sein
                       masternode 
                            |
                            | -----
                          / | \    \
                         /  |  \    kaffeemaschine usw ;-)
                        /   |   \
                     laptop |  handheld
                            |
                         media pc 
masternode ist z.b. rechner, welcher nur fuer updates zustaendig ist
Zentraler vorteil waere, dass man am knotenpunkt immer nach updates fuer software sehen kann und falls exploits da sind ev. aktionen durchfuehren wie z.b. software sperren oder wenigstens user bei verwendung warnen

security issue - bundles of software

often a software is built using different libs.

  • main program
  • libA
  • libB

(a good example would be mumble which used a speex version supplied with the mumble source instead of depending on a system lib. the problem was that the speex version mumble used was modified and because of that this had to be like this)

the source code of the whole project includes now a modified version of speex which does not appear in the packet management and if this speex version does contain a bug we won't be able to see it!

solution

  • mark all packets with similar problems as "needs manual check for bugs"
  • install the library with the packet management (create another speex packet, make that a 'ghost copy' of the original so that once a security issue is reported it is highlighted)
  • fix it upstream, do not relay on this for long

decentralized packet management

an important feature is to have the packet management not depend on a central instance. instead the main tree can be build/developed in a central fashion but can be cloned for local or offline usage.

  • think of normal packet management as svn
  • think of the concept of this work as git

security feature: checking for libs which have a bug but are still used in running programs

nowadays packet management systems do a great job with updating software against whose a security report is filled. some issues still remain as:

  • running instances still using uninstalled libs are not restarted automatically
  • the user is not informed of this
  • it is hard to track this kind of issue

rollback after a updated which did not work out

a packet management system should have a feature to make updates 'atomic'. most updates could be done with minimal downtime but if an update breaks you need to rollback to an older version. this step should be made faster. saying this i have gentoo in mind compared to debian.

on gentoo this kind of rollback can be done using binary packets. on debian this kind of action could be done pretty fast however downdates might be broken since the dependency graph calculation could be broken (can't be resolved) for some downdates after an update. this will result in a broken system!

build packets out of git/svn/cvs/others automatically

consider this situation: i use kdevelop 3.9.96 and i get a segfault using some various actions. i tell this a developer and he tells me, hey it is fixed in the current developer version.

just for testing this i would have to manually download the source. build it manually - to do this exactly as the 'gentoo portage' did can be challenging. so i would like to have two things:

  • releases are done with tags in general, no more tar.gz stuff since this can be auto generated for achieving efficiency (for example git bandwidth load issue and cpu issues).
  • a developer should be able to 'tag' a develop version which can be installed by me with ease (just as i would install kdevelop 3.9.96). i need to add a note to this one: i don't want this as a 'release replacement' because installing a developer version usually has other bugs or even broken code (which might compile, which might run but which probably will fail in it's specification).

however if a developer assists me and i just want to trigger the issue once again this would be a very helpful solution.

money

often projects have a donation option. it would be nice if some of that money could get into hosters for bandwidth and service usage. also projects could assign rewards to bug fixing...

consistency and permission check

an important feature i often would like to have is to be able to check all installed software for consistency. in case of an power loss while installing new software, how can i know that everything is alright?

another point is permissions. often software which runs as a system service has special settings as a new user which is create in the system and after the service was started as root permissons drop to an less privileged user. where is the tool that checks and/or fixes all read/write/executable/setuid bits for user/group/others?


git repo pull request feature

[04:52:48] <you> jau, genau... [04:53:21] <you> was ich bei git als default-feature genial gefunden hätte.... ne direkt in git integrierte push-request verwaltung à la GitHub bzw. Gitorious [04:53:34] <me> haha [04:53:38] <me> das waer ja lustig [04:53:48] <you> dass ein projekt einfach in seinem repo entscheidet, ob es merge-requests annimmt oder nicht... [04:53:51] <me> d.h. ich kann den maintainer des hauptzeiges zum pull bewegen [04:54:04] <me> geil [04:54:06] <you> das würde den rückfluss down → upstream bzw. user → upstream IMHO nochmal wesentlich verbessern [04:54:10] <you> genau [04:54:11] <me> sollte ich gleich in die features liste aufnehmen

Powered by MediaWiki