[1], 2011 : The first improvement enhances the original system by reducing the size of the public key from $\mathcal{\tilde{O}}(\lambda^{10})$ to $\mathcal{\tilde{O}}(\lambda^{7})$. The idea is, to publish fewer elements that can be combined on-the-fly to reach a larger public key if necessary. To achieve this, they need the already discussed optimization to set $x_0 = pq_0$, hence they settle the system on the stronger partial approximate common divisor problem.
[2], 2012 : In this paper, they further reduce the size of the public key down to $\mathcal{\tilde{O}}(\lambda^5)$. Instead of storing the large numbers $r_i+pq_i$, they only store a public seed $\text{se}$ and a set of small $\delta_i$ values that are of size $~p$. To recover the original public key, the seed $\text{se}$ is fed into the PRNG and it is $x_i = \text{PRNG}_i(\text{se}) - \delta_i$. The published $\delta_i$ are chosen that way that $$x_i = \text{PRNG}_i(\text{se}) - \delta_i \equiv r_i\pmod{q}, r_i\;\text{small}$$ The $\delta_i$ are now of the same bitsize as $p$ which is much smaller than the $x_i$ values, that were stored previously. Additionally, the apply the concept of modulus switching to the DGHV scheme. Modulus switching is a technique to reduce the noise level of a ciphertext. It does not allow infinitely many homomorphic operations, but reduces the noise grow rate from exponentially to linear (i.e. a leveled scheme), which is huge.
[3], 2013 : In this paper, the authors enhance the performance of the DGHV scheme, by converting it to a batch scheme, i.e., a scheme which allows to simultaneously process a vector of plaintexts bits. Just view this technique as some kind of parallel execution. In a simplified form, their idea is not to use a single prime $p$ as their secret key, but to use several such primes $p_0,p_1,...,p_{l-1}$. Hence their public key consists not of integers $x_i = r_i+q_ip$ but $x_i = r_i+q_i\prod^{l-1}_{j=0} p_j$. Each $p_j$ is responsible for another plaintext bit, and the ciphertext is then $$c= \text{CRT}_{p_0,...,p_{l-1}}(2r_0+m_0,...,2r_{l-1}+m_{r-1}) + q\prod^{l-1}_{j=0} p_j$$ (CRT = Chinese Remainder Theorem). Each bit can be recovered individually by computing $c$ mod $p_j$.
[4], 2014 : In this paper, the authors apply the scale-invariant property to the DGHV scheme. This technique allows get linear noise grow rate but without modulus switching. The key trick is, given a ciphertext $c$, that the message is not anymore encoded in the least significant bit of $c\;\text{mod}\;p$ but in the most significant bit of $c\;\text{mod}\;p$. A ciphertext $c$ has the form $$\text{TYPE-1:}\;\;c = r + (m+2r')\frac{p-1}{2}+qp^2$$ After multiplication of two ciphertexts $c_1$ and $c_2$ one gets a ciphertext of the form $$\text{TYPE-2:}\;\;2c_1c_2 = r'' + m_1m_2\frac{p^2-1}{2}+q'p^2$$. Then they show how publicly convert a TYPE-2 ciphertext back to TYPE-1.
Below the fold, i show how to use fixpoints of a RSA modul for homomorphic encryption.