Using newer libraries in Android project

[hamster-h]

This post is about using in your Android project libraries that are present in Android, but are of older or incompatible versions.

Assume we have an android module A and a java module J...

Assume we have an android module A and a java module J. Android module A uses this java module J, and J has some dependency D. But here is a catch: android.jar, the libraries pack that is used in android apps, already includes this dependency D, but of another version.

To be more specific: android has its internal Apache HtpComponents library of version, if I remember correctly, 4.1 with package org.apache.http. And this app uses a java SDK that uses the latest at the moment Apache HttpComponents 4.4.1. That's OK when we compile the SDK separately. But we get an error if we write this:
==========
app/build.gradle

dependencies {
compile project(':sdk')
}
----------
sdk/build.gradle

dependencies {
compile 'org.apache.httpcomponents:httpclient:4.4.1'
}
----------
Warningependency org.apache.httpcomponents:httpclient:4.4.1 is ignored for release as it may be conflicting with the internal version provided by Android.
In case of problem, please repackage it with jarjar to change the class packages
==========

Compiler tells us that he can't and won't include the newer version of library since it is already included. So, the SDK thinks that it has 4.4.1 but in fact it has 4.1. When it instantiates some class from the newer version (that was URIBuilder in my case), we get a ClassDefNotFoundException.

So, all we have to do is repackage our SDK. (Somewhere at this point I thought: compiler said what to do, and all I had to do in the end was exactly that. That's so simple, why had I thought about it so much? Why am I so dumb?)

We take the JarJar and process our SDK jar with it, renaming package org.apache.http to, for example, com.my.app.org.apache.http. Now the compiler does not say anything about duplicate packages. We launch the app... and another crash. Now that's java.lang.VerifyError:
==========
E/AndroidRuntime﹕ FATAL EXCEPTION: Thread-46989 Process: com.my.app, PID: 7732 java.lang.VerifyError: com/my/app/org/apache/http/conn/ssl/DefaultHostnameVerifier
==========

The problem is that not only Apache HttpComponents 4.4.1 is not included in android.jar, but also the whole package javax.naming. So, we have to include that in our SDK jar as well. And to include the corresponding rule for JarJar, because if we do not rename the javax.naming package, we will get an integersting error from the compiler:

the error

==========
trouble processing "javax/naming/spi/StateFactory.class":
Ill-advised or mistaken usage of a core class (java.* or javax.*)
when not building a core library.
This is often due to inadvertently including a core library file
in your application's project, when using an IDE (such as
Eclipse). If you are sure you're not intentionally defining a
core class, then this is the most likely explanation of what's
going on.
However, you might actually be trying to define a class in a core
namespace, the source of which you may have taken, for example,
from a non-Android virtual machine project. This will most
assuredly not work. At a minimum, it jeopardizes the
compatibility of your app with future versions of the platform.
It is also often of questionable legality.
If you really intend to build a core library -- which is only
appropriate as part of creating a full virtual machine
distribution, as opposed to compiling an application -- then use
the "--core-library" option to suppress this error message.
If you go ahead and use "--core-library" but are in fact
building an application, then be forewarned that your application
will still fail to build or run, at some point. Please be
prepared for angry customers who find, for example, that your
application ceases to function once they upgrade their operating
system. You will be to blame for this problem.
If you are legitimately using some code that happens to be in a
core package, then the easiest safe alternative you have is to
repackage that code. That is, move the classes in question into
your own package namespace. This means that they will never be in
conflict with core system classes. JarJar is a tool that may help
you in this endeavor. If you find that you cannot do this, then
that is an indication that the path you are on will ultimately
lead to pain, suffering, grief, and lamentation.
==========

Finally, the whole recipe:
- Use the brand new versions of libraries in java module, fear not.
- JarJar the resulting SDK jar, renaming some packages. Packages you have to rename:
-- packages of libraries that have older or incompatible versions in android;
-- packages that are absent in android.
- You will have to import (or use) not the original package (e.g., "org.apache.http") in your android app, but the renamed one (e.g., "com.my.app.org.apache.http").
- You will have to rebuild manually your SDK and jarjar it to use it in android app every time you make changes to it.