Integrate LZ4 library, compress the ramdisk

lz4/examples/blockStreaming_doubleBuffer.md

@@ -0,0 +1,109 @@
+# LZ4 Streaming API Example: Double Buffer
+by *Takayuki Matsuoka*
+
+[`blockStreaming_doubleBuffer.c`](blockStreaming_doubleBuffer.c) is an example
+of the LZ4 Streaming API where we implement double buffer (de)compression.
+
+Please note:
+
+- Firstly, read ["LZ4 Streaming API Basics"](streaming_api_basics.md).
+- This is a relatively advanced application example.
+- The output file is not compatible with lz4frame and is platform dependent.
+
+
+## What's the point of this example?
+
+The LZ4 Streaming API can be used to handle compressing a huge file in a small
+amount of memory. This example shows how to use a "Double Buffer" to compress
+blocks (i.e. chunks) of a uniform size in a stream. The Streaming API used in
+this way *always* yields a better compression ratio than the regular Block API.
+
+For an example with non-uniform blocks, see ["LZ4 Streaming API Example: Line by
+Line Text Compression"](blockStreaming_lineByLine.md).
+
+## How compression works
+
+Firstly, allocate "Double Buffer" for input and "compressed data buffer" for
+output. Double buffer has two pages, the "first" page (`Page#1`) and the "second"
+page (`Page#2`).
+
+```
+        Double Buffer
+
+      Page#1    Page#2
+    +---------+---------+
+    | Block#1 |         |
+    +----+----+---------+
+         |
+         v
+      {Out#1}
+```
+
+Next, read the first block to the double buffer's first page. Compress it with
+`LZ4_compress_continue()`. On the first compression, LZ4 doesn't have any
+previous dependencies, so it just compresses the block without dependencies and
+writes the compressed block `{Out#1}` to the compressed data buffer. After that,
+write `{Out#1}` to the file.
+
+```
+      Prefix Dependency
+
+         |         |
+         v         |
+    +---------+----+----+
+    | Block#1 | Block#2 |
+    +---------+----+----+
+                   |
+                   v
+                {Out#2}
+```
+
+Next, read the second block to the double buffer's second page and compress it.
+This time, LZ4 can use the dependency on `Block#1` to improve the compression
+ratio. This dependency is called "Prefix mode".
+
+```
+   External Dictionary Mode
+         +---------+
+         |         |
+         |         v
+    +----+----+---------+
+    | Block#3 | Block#2 |
+    +----+----+---------+
+         |
+         v
+      {Out#3}
+```
+
+Next, read the third block to the double buffer's *first* page and compress it.
+This time LZ4 can use dependency on Block#2. This dependency is called "External
+Dictionary mode".
+
+```
+      Prefix Dependency
+         +---------+
+         |         |
+         v         |
+    +---------+----+----+
+    | Block#3 | Block#4 |
+    +---------+----+----+
+                   |
+                   v
+                {Out#4}
+```
+
+Continue this procedure till the end of the file.
+
+
+## How decompression works
+
+Decompression follows the reverse order:
+
+- Read the first compressed block.
+- Decompress it to the first page and write that page to the file.
+- Read the second compressed block.
+- Decompress it to the second page and write that page to the file.
+- Read the third compressed block.
+- Decompress it to the *first* page and write that page to the file.
+
+Continue this procedure till the end of the compressed file.