Release metrics-lib version 1.0.0

added component::metrics/library owner::karsten priority::medium resolution::fixed severity::normal status::closed type::enhancement labels

I'd like to move forward here, because having a release and package blocks other tasks like packaging applications depending on metrics-lib.

That's why I made an unofficial Debian package for metrics-lib, mostly to find out what's missing in metrics-lib or what needs to be improved for that.

I'd want to hear back from Debian packagers how that package could be improved and what changes should be made upstream in metrics-lib.

The end result could be an improved upstream build and release process and, as a welcome side product, an (unofficial) Debian package.

Here's how I built and installed the package:

https://github.com/kloesing/metrics-lib-debian

Thoughts?

Trac:
Sponsor: N/A to N/A
Severity: N/A to Normal

Great that this issue is moving forward now! And, the debian package is a good idea.

The Readme.md suggests installing jdk6 which could cause problems in case a higher jdk version is already installed. There might be a way to just specify a minimal jdk version? As I'm not a debian packager I do not have a good suggestion here.

iwakeh, the README.md was mostly there to make it easy to reproduce the steps I did for building package. Or really, they were there as note to myself. These instructions wouldn't go into a real package repository. I just created an issue on GitHub to make that clearer. Thanks for the feedback!

So, after we have a working (unofficial, unpublished) Debian package, I sat down and wrote a first draft of a contributor's guide to how we develop metrics-lib. That guide also contains suggestions for a change log and for putting out releases. I'd appreciate feedback on that, or even patches! Thanks!

Pushed CONTRIB.md to master. What else is missing in the release process? If there's nothing, I'd just put out the first release. Thoughts?

It might be good to also mention the used versions of the two dependencies at least in CONTRIB.md (they are not listed in build.xml (yet?))

Maybe, add to line 44:

Every public interface or method should have a comment ...
Every new public interface or method should have a javadoc comment ...

the rest of the sentence explaines that we don't have it yet.

Why is there no third component for the version? It might be useful for troubleshooting and bugfixing to know exactly the version that caused the problem. Unintended side-effects when cleaning the code base or doing other minor changes could be more easily detected. Or, when keeping the two number versioning, maybe add the git revision of the particular release to META-INF.

Replying to iwakeh:

It might be good to also mention the used versions of the two dependencies at least in CONTRIB.md (they are not listed in build.xml (yet?))

Good point, I just added version numbers to CONTRIB.md and to build.xml.

Maybe, add to line 44: {{{ Every public interface or method should have a comment ... Every new public interface or method should have a javadoc comment ... }}} the rest of the sentence explaines that we don't have it yet.

Hmm, I'm not sure what you have in mind. Can you paste the paragraph you want here, and I'll copy it over?

Why is there no third component for the version? It might be useful for troubleshooting and bugfixing to know exactly the version that caused the problem. Unintended side-effects when cleaning the code base or doing other minor changes could be more easily detected. Or, when keeping the two number versioning, maybe add the git revision of the particular release to META-INF.

I don't feel strongly about either using two or three version components. For Onionoo, it made sense to have three components (major protocol changes, minor protocol changes, implementation changes). I only came up with two components (major changes, minor changes), but I'd say if we can come up with some reasonable guidelines for when we'd raise the three version components for metrics-lib, let's have three of them. What would such guidelines be?

Thanks!

Thanks for adding the dependency versions!

I attach a diff for CONTRIB.md containing the threefold version description and the change from my earlier comment, which really was a little unclear.

In addition, I changed the URLs for the dependencies downloads, as the mentioned dependency versions require some searching.

=== btw The dependencies could be updated: junit stable is 4.12 and commons-compress should not cause a problem up to 1.9.0 (1.10 is not backward compatible anymore) This could be a new issue?

Trac:
contrib.diff

Thanks for the edits! I just merged them with a few trivial changes, including taking out the more detailed links that you put in, because I'm afraid they'll break could rather sooner than later. The versioning schema you suggest sounds reasonable, let's use it.

Regarding the newer library versions, we still depend on Debian wheezy (oldstable) for things running on Tor VMs, which is why I'd rather not want to upgrade to newer versions just yet. But you're right that using them shouldn't cause a problem. Should we say in CONTRIB.md that JUnit can be 4.10 to 4.12 and Commons Compress 1.4.1 to 1.9.0?

Please find branch task-13166 in my public repository for some more commits that add a change log and an Ant task to generate a release tarball. Please suggest any changes that need to happen before actually putting out 1.0.0. Thanks!

Trac:
Status: new to needs_review

Quick list of nice-to-haves for a release:

a jar manifest that contains at least: git revision, copyright, creator
what about signing the jar using ant task signjar?
I'd expect sources in a release tarball that I download. And, if they are not huge also the dependencies used for building the binaries. The two commons libs are small.

Maybe, add a simple explanation to the download links pointing out where to roughly find the downloads i.e. one needs to look for the archives or at least make clear that the provided links only link to the respective projects and not the libs.

The dependency versions should just refer to the dependency versions used, unless other versions were explicitly tested, I think.

Replying to iwakeh:

Quick list of nice-to-haves for a release:

a jar manifest that contains at least: git revision, copyright, creator

I included a few fields in the manifest, but I didn't find how to include the Git revision. Any ideas?

what about signing the jar using ant task signjar?

My plan was to upload an .asc file along with the tarball. Is that a bad plan?

I'd expect sources in a release tarball that I download. And, if they are not huge also the dependencies used for building the binaries. The two commons libs are small.

Good point. But should we put out two different tarballs, one with sources and one with the binary .jar file? Would you be able to extend the tar target (maybe the name should be dist?) to also produce a source tarball?

Maybe, add a simple explanation to the download links pointing out where to roughly find the downloads i.e. one needs to look for the archives or at least make clear that the provided links only link to the respective projects and not the libs.

There, added a quick description.

The dependency versions should just refer to the dependency versions used, unless other versions were explicitly tested, I think.

Okay.

Please find my updated branch.

Good that you added the version, too.

For the git revision I'd use an exec ant task and in there the outproperty (or similar) for the result of calling git rev-parse HEAD or something along these lines.

The signature is fine for the entire tarball.

Maybe provide the plain library (signed) jar-file w/o a surrounding tar and the full compressed tar file.

Good ideas. I think I applied them all, plus a few more. Please take another look at the updated branch, and please give it a quick try. Thanks so much for your feedback here!

Any remaining thoughts or concerns on my branch that I updated last week? If not, I'll release version 1.0.0 tomorrow. :)

The jar and tar ant tasks work fine.

I still would like a signed jar even for the sources, because you have each file inside the jar hashed and the signature inside. Also a step toward the deterministic build (at least with additional measures). It's so easy with java.

For the sources alone the signed jar format is also better even if external libs are added, which would be nice for people starting to work with metrics-libs.

Sorry that I didn't state these concerns more clearly earlier.

Okay, let's see. I just added another commit that produces a sources jar and that signs both jars. Please take a look at my updated branch.

I also produced signed jars and a signed tarball for testing purposes only. Here's how you can try them out:

wget https://people.torproject.org/~karsten/volatile/descriptor-1.0.0.jar
wget https://people.torproject.org/~karsten/volatile/descriptor-1.0.0-sources.jar
wget https://people.torproject.org/~karsten/volatile/descriptor-1.0.0-sources.tar.bz2
wget https://people.torproject.org/~karsten/volatile/descriptor-1.0.0-sources.tar.bz2.asc
wget https://people.torproject.org/~karsten/volatile/karsten.cer
wget https://people.torproject.org/~karsten/volatile/karsten.cer.asc
gpg --keyserver pgp.mit.edu --recv-key 0x4EFD4FDC3F46D41E
gpg --verify karsten.cer.asc
keytool -importcert -alias karsten -file karsten.cer
jarsigner -verify descriptor-1.0.0.jar
jarsigner -verify descriptor-1.0.0-sources.jar
gpg --verify descriptor-1.0.0-sources.tar.bz2.asc

Regarding external libraries, those are not contained in either jars or tar. Should we fix that? And if so, why?

As always, thanks!

Great! The verifying works fine.

Well, adding the external libs to the tar (and maybe also signing the external jars) would make explicit what external software was used. And with the hashes in the signed jars one could verify, if these are the same as those available from other sources, on a class file level. That's the tin-hatty reasoning.

The second reason is convenience: I can just download the tar and have everything - verified by the Torproject - in place for deploying descriptor in my project and/or for starting development on it. This is just one simple download instead of searching for the deps in the web or adding packets to my linux system. A tar with everything (signed class jar, sources jar, external jars) will provide this.

Okay, I tried to incorporate all suggestions, including adding our own jars and external jars to the tarball, but I refrained from signing external jars for two reasons: they're already signed as part of the tarball, and I don't want to modify them by adding a signature. It's also still possible to compare external tarballs file by file, in case its overall digest is different from what you get from other sources.

Mind taking another look?

https://people.torproject.org/~karsten/volatile/descriptor-1.0.0.tar.gz

https://people.torproject.org/~karsten/volatile/descriptor-1.0.0.tar.gz.asc

https://gitweb.torproject.org/user/karsten/metrics-lib.git/log/?h=task-13166

Thanks!

All is fine. Good that you added hamcrest, too.

Thanks for all this work for the first release!

I documented the signing and verification process here and somehow pushed it directly to master without asking for review here. Oops. Please send patches anyway. Otherwise it'll go into the release soon. Thanks!

And now it's released. Yay! Thanks for all the help! Resolving.

Trac:
Status: needs_review to closed
Resolution: N/A to fixed

closed

mentioned in issue #13824 (moved)

mentioned in issue #16746 (moved)

mentioned in issue #17370 (moved)

Release metrics-lib version 1.0.0

Child items ...

Activity