Post new topic   Reply to topic
View previous topic Printable version Log in to check your private messages View next topic
Author Message
LatOffline
15 Post subject: Unable to find expected entry 'Packages' in Release file  PostPosted: 21.08.2011, 21:50



Joined: 2010-09-19
Posts: 164

Status: Offline
After upgrading apt 0.8.15.6
I get an error when I run apt-get update
      Code:
W: Failed to fetch http://localhost/repo/binary/./Release  Unable to find expected entry 'Packages' in Release file (Wrong sources.list entry or malformed file)

Downgrading to 0.8.15.5 makes everything to work as it has been for years.

My local repo tree is
..../repo/binary/bunch-of-homemade.deb
..../repo/mirror/bunch-of-officialdebs-that-I-Want-to-keep-localy.deb

I update the repo with the next script residing in ..../repo/
      Code:
#!/bin/bash

dpkg-scanpackages -m binary override | gzip -9c > binary/Packages.gz
dpkg-scanpackages  -m binary override | bzip2 -  > binary/Packages.bz2

echo " binary Packages updated"

dpkg-scansources  source override | gzip -9c > source/Sources.gz
dpkg-scansources   source override | bzip2 -  > source/Sources.bz2

dpkg-scanpackages -m mirror mirror-override | bzip2 - > mirror/Packages.bz2

echo " source and mirror Packages updated"

# run the below once in a while
#apt-ftparchive contents binary | gzip -9c > binary/Contents.gz
apt-ftparchive contents . | gzip -9c > Contents.gz
echo " Contents done"
apt-ftparchive release . > Release

echo " Done updating Release"

gpg --sign -ba --output Release.gpg Release

echo " Done Signing Release"

echo " Repo update process done"

#-a, --arch arch
#         Use a pattern consisting of *_all.deb and *_arch.deb instead  of  scanning  for all debs

"override" file has the names of all .debs in binary/ e.g.:
      Code:
powerpanel extra unstable
powerpanel-lsb extra unstable
lataliases extra unstable
grub2-memtest-symblink extra unstable
etc

" mirror-override" is a similar file for mirror/

the source.list looks like this
      Code:
cat /etc/apt/sources.list.d/lat.list
# servidor
#deb http://localhost/repo/ binary/
deb http://localhost/repo/binary ./

#deb http://localhost/repo2 binary/
deb http://localhost/repo/ mirror/
#deb-src http://localhost/repo/ source/


I have apache installed so the rest of the computers in the network can apt-get them
 
 View user's profile Send private message  
Reply with quote Back to top
DonKultOffline
Post subject: RE: Unable to find expected entry  PostPosted: 22.08.2011, 17:55
Team Member


Joined: 2010-09-02
Posts: 482

Status: Offline
Thanks for your detailed bugreport!
(feel free to sent them to the debian bugtracker with 'reportbug' next time, more audience in general and better tracking -- other can see that you found a problem, too.)

I have a guess: Your script indicates that you create signatures for your repository, but you haven't added the key to apt trusted.gpg, did you?


The error message tells you a valid point: Your Release file doesn't include the hashsum for the file called 'Packages' -- only for the compressed Packages.gz and Packages.bz2, so APT can validate that it downloaded the correct file, but it can't check if it was correctly extracted.

A comment on your script:
Why do you use dpkg-scanpackage? apt-ftparchive should do all of this just fine and compared to your scripted implementation in a more robust way - it e.g creates all compressed files in one go. Have a look at the 'generate' action of apt-ftparchive.

And on your sources.list:
APT supports a file:// method to access stuff directly from disk which e.g. removes the need to 'download' the *.deb files from localhost to store it in /var/cache/apt/archives.

_________________
MfG. DonKult
"I never make stupid mistakes. Only very, very clever ones." ~ The Doctor
 
 View user's profile Send private message Visit poster's website  
Reply with quote Back to top
LatOffline
Post subject: Re: RE: Unable to find expected entry  PostPosted: 23.08.2011, 01:54



Joined: 2010-09-19
Posts: 164

Status: Offline
      DonKult wrote:
I have a guess: Your script indicates that you create signatures for your repository, but you haven't added the key to apt trusted.gpg, did you?
Correct. I haven't added it to trusted.gpg. Reason: idk how to do it XD

      DonKult wrote:
The error message tells you a valid point: Your Release file doesn't include the hashsum for the file called 'Packages' -- only for the compressed Packages.gz and Packages.bz2, so APT can validate that it downloaded the correct file, but it can't check if it was correctly extracted.

A comment on your script:
Why do you use dpkg-scanpackage? apt-ftparchive should do all of this just fine and compared to your scripted implementation in a more robust way - it e.g creates all compressed files in one go. Have a look at the 'generate' action of apt-ftparchive.
I built my repo from scratches of google searches. I'll read man apt-ftparchive generate

      DonKult wrote:
And on your sources.list:
APT supports a file:// method to access stuff directly from disk which e.g. removes the need to 'download' the *.deb files from localhost to store it in /var/cache/apt/archives.
I didn't realized that file:// avoids "downloading" the debs. I do remember reading you could use file://, but I preferred to use http:// because I only needed to scp the file to the rest of the pcs in the network and just replace localhost. I will change the protocol, tho
 
 View user's profile Send private message  
Reply with quote Back to top
LatOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 23.08.2011, 01:58



Joined: 2010-09-19
Posts: 164

Status: Offline
If I use apt-ftparchive genereate, I have to have this tree? http://www.debian.org/doc/manuals/repos ... ry-example
 
 View user's profile Send private message  
Reply with quote Back to top
LatOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 23.08.2011, 02:26



Joined: 2010-09-19
Posts: 164

Status: Offline
You were right. I modified my script a bit while I get around with apt-ftparchive generate
      Code:
#!/bin/bash

#dpkg-scanpackages -m binary override | gzip -9c > binary/Packages.gz
#dpkg-scanpackages -m binary override | bzip2 -  > binary/Packages.bz2
dpkg-scanpackages -m binary override > binary/Packages

echo " binary Packages updated"

#dpkg-scansources  source override | gzip -9c > source/Sources.gz
#dpkg-scansources  source override | bzip2 -  > source/Sources.bz2
dpkg-scansources  source override  > source/Sources

#dpkg-scanpackages -m mirror mirror-override | bzip2 - > mirror/Packages.bz2
dpkg-scanpackages -m mirror mirror-override > mirror/Packages

echo " source and mirror Packages updated"

# run the below once in a while
#apt-ftparchive contents binary | gzip -9c > binary/Contents.gz
apt-ftparchive contents . | gzip -9c > Contents.gz
echo " Contents done"
apt-ftparchive release . > Release
apt-ftparchive release binary > binary/Release
apt-ftparchive release mirror > mirror/Release

echo " Done updating Release"

gpg --sign -ba --output Release.gpg Release

echo " Done Signing Release"

echo " Repo update process done"

#-a, --arch arch
#         Use a pattern consisting of *_all.deb and *_arch.deb instead  of  scanning  for all debs
Now the error is gone with apt 0.8.15.6
 
 View user's profile Send private message  
Reply with quote Back to top
DonKultOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 23.08.2011, 09:10
Team Member


Joined: 2010-09-02
Posts: 482

Status: Offline
With a config like this you should be able to replace most of your script with apt-ftparchive generate this.config:
      Code:
Default
{
   Packages::Compress ". gzip"; # the dot stands for no compression
   Sources::Compress ". gzip"; # 'bzip2', 'lzma' and/or 'xz' can be added, too
   Contents::Compress ". gzip";
}
bindirectory "binary"
{
   Packages "binary/Packages";
   Contents "binary/Contents";
   #BinOverride "override" # commented out as i don't know if you really use/need a override file
}
bindirectory "source"
{
   Sources "source/Sources";
}
bindirectory "mirror"
{
   Packages "mirror/Packages";
   Contents "mirror/Contents";
}

Haven't tested it through.


Regarding adding your key to the trusted database:
It's kind of pointless for file:// usage, but in case you have another repo:
      Code:
$ gpg --export yourKeyID > /tmp/my.key
# touch /etc/apt/trusted.gpg.d/whatever.gpg
# apt-key --keyring /etc/apt/trusted.gpg.d/whatever.gpg add /tmp/my.key

replace 'whatever' with a good name describing the archive - the name itself is not important for APT, it just needs the '.gpg' extension. yourKeyID should be kind of obvious - i assume its already present on your machine in the user-keyring, if not you need to --recv-key from the net. The touch is needed, otherwise gpg doesn't create the keyring, i will have a look at removing this silliness later…

The security-guy in me wants to add that you should validate that the key is really your key (especially if you downloaded it from the internet) and that storing a file in /tmp/ isn't save on a multi-user system.

_________________
MfG. DonKult
"I never make stupid mistakes. Only very, very clever ones." ~ The Doctor
 
 View user's profile Send private message Visit poster's website  
Reply with quote Back to top
piperOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 23.08.2011, 23:05
Moderator


Joined: 2010-09-11
Posts: 480
Location: cheektowaga, ny
Status: Offline
A good gpg cheat sheet that I use

http://irtfweb.ifa.hawaii.edu/~lockhart/gpg/gpg-cs.html

Setup your own repo

http://wiki.debian.org/HowToSetupADebianRepository

Creating your own Signed APT Repository and Debian Packages

http://scotbofh.wordpress.com/2011/04/2 ... -packages/

All came in handy for me Wink

_________________
debian sid | apt-get into it
 
 View user's profile Send private message  
Reply with quote Back to top
LatOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 24.08.2011, 01:51



Joined: 2010-09-19
Posts: 164

Status: Offline
Awesome. I can apt-get update locally and from other pc without errors. ArchiveDir was missing, so the working .conf is
      Code:
Dir {
   ArchiveDir "/path/to/repo"
}
Default
{
   Packages::Compress ". gzip"; # the dot stands for no compression
   Sources::Compress ". gzip"; # 'bzip2', 'lzma' and/or 'xz' can be added, too
   Contents::Compress ". gzip";
}
bindirectory "binary"
{
   Packages "binary/Packages";
   Contents "binary/Contents";
   #BinOverride "override" # commented out as i don't know if you really use/need a override file
}
bindirectory "source"
{
   Sources "source/Sources";
}
bindirectory "mirror"
{
   Packages "mirror/Packages";
   Contents "mirror/Contents";
}
No more needs for overrides
Thank you DonKult
 
 View user's profile Send private message  
Reply with quote Back to top
DonKultOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 24.08.2011, 15:55
Team Member


Joined: 2010-09-02
Posts: 482

Status: Offline
For the record as this isn't exactly clear in this thread: 'apt-ftparchive generate' doesn't generate Release files, so this needs to remain in the script.

(We should add that at some point…)

_________________
MfG. DonKult
"I never make stupid mistakes. Only very, very clever ones." ~ The Doctor
 
 View user's profile Send private message Visit poster's website  
Reply with quote Back to top
LatOffline
Post subject: RE: Re: RE: Unable to find expected entry  PostPosted: 24.08.2011, 19:26



Joined: 2010-09-19
Posts: 164

Status: Offline
I was wondering that myself.
Ok. Script is now:
      Code:
#!/bin/bash
apt-ftparchive generate repo.conf
apt-ftparchive release binary > binary/Release
apt-ftparchive release mirror > mirror/Release

I need one for the root, don't it?
      Code:
apt-ftparchive release . > Release
 
 View user's profile Send private message  
Reply with quote Back to top
DonKultOffline
Post subject: Re: RE: Re: RE: Unable to find expected entry  PostPosted: 24.08.2011, 22:29
Team Member


Joined: 2010-09-02
Posts: 482

Status: Offline
      Lat wrote:
I need one for the root, don't it?
      Code:
apt-ftparchive release . > Release

No, it's never accessed because you are accessing your repositories (yes, you have three!) with binary and mirror (plus source as the third repo). That these independent repositories share the same parent-directory is nothing APT cares about. So you just need to create the Release files in these subdirectories as it's the repository which is released and not it's parent-directory. And you should sign each of these Release files to make it look really professional. Smile
Have a look at 'man apt-secure' to create really shinny Release.gpg and InRelease files. Wink

_________________
MfG. DonKult
"I never make stupid mistakes. Only very, very clever ones." ~ The Doctor
 
 View user's profile Send private message Visit poster's website  
Reply with quote Back to top
Display posts from previous:     
Jump to:  
All times are GMT - 12 Hours
Post new topic   Reply to topic
View previous topic Printable version Log in to check your private messages View next topic
Powered by Zafenio