-where MSE is the mean squared error.
-
-
-The Libnetpbm Netpbm Image -Processing Manual describes the the libnetpbm functions for +Processing Manual describes the libnetpbm functions for processing image data.
The Libnetpbm Utility Manual diff -urNp old/userguide/pamfunc.html new/userguide/pamfunc.html --- old/userguide/pamfunc.html 2017-09-05 14:20:09.738743398 +0200 +++ new/userguide/pamfunc.html 2017-09-05 14:47:41.760644848 +0200 @@ -57,7 +57,7 @@ output image. and bit string (such as and with 01001000). For the arithmetic functions, the function arguments and results are the fraction that a sample is of the maxval, i.e. normal interpretation of PAM tuples. But for the bit string -functions, the value is the the bit string whose value as a binary cipher is +functions, the value is the bit string whose value as a binary cipher is the sample value, and the maxval indicates the width of the bit string.
Sizes are in points, as a floating point number. diff -urNp old/userguide/pbmtog3.html new/userguide/pbmtog3.html --- old/userguide/pbmtog3.html 2017-09-05 14:20:09.735743403 +0200 +++ new/userguide/pbmtog3.html 2017-09-05 14:48:55.648255793 +0200 @@ -77,7 +77,7 @@ You cannot specify both.
Before Netpbm 10.79 (June 2017), there was a different program by the same -name in Netpbm, which was written by by Paul Haeberli +name in Netpbm, which was written by Paul Haeberli <paul@manray.sgi.com> in 1989 and then modified extensively by others. diff -urNp old/userguide/ppmtompeg.html new/userguide/ppmtompeg.html --- old/userguide/ppmtompeg.html 2017-09-05 14:20:09.739743396 +0200 +++ new/userguide/ppmtompeg.html 1970-01-01 01:00:00.000000000 +0100 @@ -1,1291 +0,0 @@ - - -
-This program is part of Netpbm. - -
ppmtompeg produces an MPEG-1 video stream. MPEG-1 is the -first great video compression method, and is what is used in Video CDs -(VCD). ppmtompeg originated in the year 1995. DVD uses a more -advanced method, MPEG-2. There is an even newer method called MPEG-4 -which is also called Divx. I don't know where one finds that used. - -
There's technically a difference between a compression method for -video and an actual file (stream) format for a movie, and I don't know -if it can be validly said that the format of the stream -ppmtompeg produces is MPEG-1. - -
Mencoder from the Mplayer -package is probably superior for most video format generation -needs, if for no other reason than that it is more popular. - -
The programming library PM2V -generates MPEG-2 streams. - -
Use Mplayer (not part of Netpbm) -to do the reverse conversion: to create a series of PNM files from an MPEG -stream. - -
param_file is a parameter file which includes a list of -input files and other parameters. The file is described in detail -below. - -
To understand this program, you need to understand something about -the complex MPEG-1 format. One source of information about this -standard format is the section Introduction to MPEG in the Compression FAQ. - -
The -gop, -combine_gops, -frames, and --combine_frames options are all mutually exclusive. - -
These statistics include how many I, P, and B frames there were, -and information about compression and quality. - - -
Thus, unless you're mixing and matching GOP files from different -sources, you can simply use the same parameter file for creating the -GOP files (-gop) and for later turning them into an MPEG stream -(-combine_gops). - - -
Use ppmtompeg -combine_frames to combine these frames later into -an MPEG stream. - - -
The parameter file may specify input frame files in the same manner -as normal input files -- except instead of using INPUT_DIR, INPUT, and -END_INPUT, use FRAME_INPUT_DIR, FRAME_INPUT, and FRAME_END_INPUT. If -no input frame files are specified, then the default is to use the -output file name with suffix .frame.frame_num, with -frame_num starting from 0, as the input files. - - - -
The output is in the form of a matrix, each entry corresponding to one -motion vector in the search window. The center of the matrix -represents (0,0) motion vectors. - -
The parameter file must contain the following -lines (except when using the -combine_gops or -combine_frames -options): - -
- PATTERN IBBPBBPBBPBBPBB -- -
See I Frames, P Frames, B Frames. - -
To have ppmtompeg read all the frames serially from Standard -Input, specify -
- INPUT_DIR stdin -- -
There are three types of lines between INPUT and END_INPUT. First, -a line may simply be the name of an input file. Second, the line -may be of the form single_star_expr -[x-y]. -single_star_expr can have a single * in it. It is -replaced by all the numbers between x and y inclusive. So, for -example, the line tennis*.ppm [12-15] refers to the files -tennis12.ppm, tennis13.ppm, tennis14.ppm, tennis15.ppm. - -
Uniform zero-padding occurs, as well. For example, the line -football.*.ppm [001-130] refers to the files football.001.ppm, -football.002.ppm, ..., football.009.ppm, football.010.ppm, ..., -football.130.ppm. - -
The third type of line is: single_star_expr -[x-y+s], where the -line is treated exactly as above, except that we skip by s. Thus, the -line football.*.ppm [001-130+4] refers to the files -football.001.ppm, football.005.ppm, football.009.ppm, -football.013.ppm, etc. - -
Furthermore, a line may specify a shell command to execute to -generate lines to be interpreted as described above, as if those lines -were in the parameter file instead. Use back ticks, like in the -Bourne Shell, like this: - -
- `cat myfilelist` -- -
-If input is from Standard Input (per the INPUT_DIR statement), -ppmtompeg ignores the INPUT/END_INPUT block, but -it still must be present. - -
- INPUT_CONVERT * -- -
Otherwise, conversion_command is a shell command that causes -an image in the format your specified with BASE_FILE_FORMAT to -be written to Standard Output. ppmtompeg executes the command -once for each line between INPUT and END_INPUT (which is -normally, but not necessarily, a file name). In the conversion -command, ppmtompeg replaces each '*' with the contents of that -line. - - If you had a bunch of gif files, you might say: -
- INPUT_CONVERT giftopnm * -- - If you have a bunch of separate a.Y, a.U, and a.V files (where - the U and V have already been subsampled), then you might say: - -
- INPUT_CONVERT cat *.Y *.U *.V -- -
Input conversion is not allowed with input from stdin, so use - -
- INPUT_CONVERT * -- -as described above. - -
width and height are the width and height of each -frame in pixels. - -
When ppmtompeg can get this information from the input image -files, it ignores the SIZE parameter and you may omit it. - -
When the image files are in YUV format, the files don't contain -dimension information, so SIZE is required. - -
When ppmtompeg is running in parallel mode, not all of the -processes in the network have access to the image files, so -SIZE is required and must give the same dimensions as the -input image files. - -
Normally, it makes sense to make your GOP size a multiple of your -pattern length (the latter is determined by the PATTERN parameter file -statement). - -
See Group Of Pictures. - -
| Reference | -Compression | -Speed | -Quality I | -Quality P | -Quality B | -
|---|---|---|---|---|---|
| Decoded | -1000 | -1000 | -1000 | -969 | -919 | -
| Original | -885 | -1373 | -1000 | -912 | -884 | -
The following lines are optional: - -
Before Netpbm 10.26 (January 2005), ppmtompeg would drop -trailing B frames from your movie, since a movie can't end with a B -frame. (See I Frames, P Frames, B Frames. -You would have to specify FORCE_ENCODE_LAST_FRAME to stop -that from happening and get the same function that ppmtompeg -has today. - - -
-The 8 lines immediately following NIQTABLE specify the quantization -table. Each line defines a table row and consists of 8 integers, -whitespace-delimited, which define the table columns. - -
ratio must be 1.0, 0.6735, 0.7031, 0.7615, 0.8055, 0.8437, -0.8935, 0.9157, 0.9815, 1.0255, 1.0695, 1.0950, 1.1575, or 1.2015. - -
rate must be 23.976, 24, 25, 29.97, 30, 50, 59.94, or 60. - -
rate must be an integer. - -
A Video Verifying Buffer is a buffer in which a decoder keeps the -decoded bits in order to match the uneven speed of the decoding with -the required constant playback speed. - -
As ppmtompeg encodes the image, it simulates the decoding -process in terms of how many bits would be in the VBV as each frame gets -decoded, assuming a VBV of the size you indicate. - -
If you specify the WARN_VBV_UNDERFLOW statement, -ppmtompeg issues a warning each time the simulation underflows -the buffer, which suggests that an underflow would occur on playback, -which suggests the buffer is too small. - -
If you specify the WARN_VBV_OVERFLOW statement, -ppmtompeg issues a warning each time the simulation overflows -the buffer, which suggests that an overflow would occur on playback, -which suggests the buffer is too small. - -
These options were new in Netpbm 10.26 (January 2005). Before that, -ppmtompeg issued the warnings always. - -
If you specify FORCE_I_ALIGN, ppmtompeg will increase -the test frames value enough to maintain the alignment. - -
If there aren't enough frames for every slave to have the indicated -number of test frames, ppmtompeg will give some slaves fewer. - - -
Smaller values of t increase communication, but improve load -balancing. The default is 30 seconds. - -
You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, -and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. - -
You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, -and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. - - -
This has the advantage of minimal scheduling overhead. Where slaves -have different speeds, though, it makes inefficient use of the fast -ones. Where slaves are the same speed, it also has the disadvantage -that they all finish at the same time and feed their output to the -single Combine Server in a burst, which makes less efficient use of -the Combine Server and thus can increase the total elapsed time. - -
You may specify only one of PARALLEL_TIME_CHUNKS, PARALLEL_CHUNK_TAPER, -and PARALLEL_PERFECT. PARALLEL_CHUNK_TAPER is usually best. - -
Be sure to set up .rhosts files or SSH key authorizations -where needed. Otherwise, you'll have to type in passwords. - -
On some HP machines, rsh is the restricted shell, and you want -to specify remsh. - -
This document used to say there was an argument to -FORCE_I_ALIGN which was the number of frames ppmtompeg -would use (and was required to be a multiple of the pattern length). -But ppmtompeg has apparently always ignored that argument, and -it does now. - -
This is mostly useful for debugging. - -
This works only if you're using a shared filesystem to communicate -between the servers. - -
This option was new in Netpbm 10.26 (January 2005). - -
If you use the -combine_gops option, then you need to specify -only the SIZE and OUTPUT values in the parameter file. In -addition, the parameter file may specify input GOP files in the same -manner as normal input files -- except instead of using INPUT_DIR, -INPUT, and END_INPUT, use GOP_INPUT_DIR, GOP_INPUT, and GOP_END_INPUT. -If you specify no input GOP files, then ppmtompeg uses by default the -output file name with suffix .gop.gop_num, with gop_num -starting from 0, as the input files. - -
If you use the -combine_frames option, then you need to -specify only the SIZE, GOP_SIZE, and OUTPUT values in the -parameter file. In addition, the parameter file may specify input -frame files in the same manner as normal input files -- except instead -of using INPUT_DIR, INPUT, and END_INPUT, use FRAME_INPUT_DIR, -FRAME_INPUT, and FRAME_END_INPUT. If no input frame files are -specified, then the default is to use the output file name with suffix -.frame.frame_num, with frame_num starting from 0, -as the input files. - -
Any number of spaces and tabs may come between each option and value. Lines -beginning with # are ignored. Any other lines are ignored except for -those between INPUT and END_INPUT. This allows you to use the same -parameter file for normal usage and for -combine_gops and --combine_frames. - -
The file format is case-sensitive so all keywords should be in -upper case. - -
The statements may appear in any order, except that the order within -a block statement (such as INPUT ... END INPUT) is significant. - -
ppmtompeg is prepared to handle up to 16 B frames between -reference frames when encoding with input from stdin. (To build a -modified ppmtompeg with a higher limit, change the constant -B_FRAME_RUN in frame.c and recompile). - -
The quantization scale values (qscale) give a trade-off between -quality and compression. Using different Qscale values has very little -effect on speed. The qscale values can be set separately for I, P, and -B frames. - -
You select the qscale values with the IQSCALE, -PQSCALE, and BSCALE parameter file statements. - -
A qscale value is an integer from 1 to 31. Larger numbers give
-better compression, but worse quality. In the following, the quality
-numbers are peak signal-to-noise ratio, defined as:
-
-where MSE is the mean squared error.
-
-
-
Flower garden tests: - -
| Qscale | -I Frames | -P Frames | -B Frames | -
|---|---|---|---|
| 1 | -43.2 | -46.3 | -46.5 | -
| 6 | -32.6 | -34.6 | -34.3 | -
| 11 | -28.6 | -29.5 | -30.0 | -
| 16 | -26.3 | -26.8 | -28.6 | -
| 21 | -24.7 | -25.0 | -27.9 | -
| 26 | -23.5 | -23.9 | -27.5 | -
| 31 | -22.6 | -23.0 | -27.3 | -
| Qscale | -I Frames | -P Frames | -B Frames | -
|---|---|---|---|
| 1 | -2 | -2 | -2 | -
| 6 | -7 | -10 | -15 | -
| 11 | -11 | -18 | -43 | -
| 16 | -15 | -29 | -97 | -
| 21 | -19 | -41 | -173 | -
| 26 | -24 | -56 | -256 | -
| 31 | -28 | -73 | -330 | -
There are several different motion vector search techniques -available. There are different techniques available for P frame -search and B frame search. Using different search techniques present -little difference in quality, but a large difference in compression -and speed. - -
There are 4 types of P frame search: Exhaustive, TwoLevel, -SubSample, and Logarithmic. - -
There are 3 types of B frame search: Exhaustive, Cross2, and -Simple. - -The recommended search techniques are TwoLevel and Logarithmic for -P frame search, and Cross2 and Simple for B frame search. Here are -some numbers comparing the different search methods: - -
| Technique | -Compression1 | -Speed 2 | -Quality 3 | -
|---|---|---|---|
| Exhaustive | -1000 | -1000 | -1000 | -
| SubSample | -1008 | -2456 | -1000 | -
| TwoLevel | -1009 | -3237 | -1000 | -
| Logarithmic | -1085 | -8229 | -998 | -
| Technique | -Compression1 | -Speed2 | -Quality3 | -
|---|---|---|---|
| Exhaustive | -1000 | -1000 | -1000 | -
| Cross2 | -975 | -1000 | -996 | -
| Simple | -938 | -1765 | -991 | -
For some reason, Simple seems to give better compression, but it -depends on the image sequence. - -
Select the search techniques with the PSEARCH_ALG and -BSEARCH_ALG parameter file statements. - - - -
A Group of Pictures (GOP) is a roughly independently decodable -sequence of frames. An MPEG video stream is made of one or more -GOP's. You may specify how many frames should be in each GOP with the -GOP_SIZE parameter file statement. A GOP always starts with an -I frame. - -
Instead of encoding an entire sequence, you can encode a single -GOP. To do this, use the -gop command option. You can later -join the resulting GOP files at any time by running ppmtompeg -with the -combine_gops command option. - - -
A slice is an independently decodable unit in a frame. It can be -as small as one macroblock, or it can be as big as the entire frame. -Barring transmission error, adding slices does not change quality or -speed; the only effect is slightly worse compression. More slices are -used for noisy transmission so that errors are more recoverable. Since -usually errors are not such a problem, we usually just use one slice -per frame. - -
Control the slice size with the SLICES_PER_FRAME parameter -file statement. - -
Some MPEG playback systems require that each slice consist of whole -rows of macroblocks. If you are encoding for this kind of player, if -the height of the image is H pixels, then you should set the -SLICES_PER_FRAME to some number which divides H/16. For example, if -the image is 240 pixels (15 macroblocks) high, then you should use -only 15, 5, 3, or 1 slices per frame. - -
Note: these MPEG playback systems are really wrong, since the MPEG -standard says this doesn't have to be so. - - - -
The search window is the window in which ppmtompeg searches -for motion vectors. The window is a square. You can specify the size -of the square, and whether to allow half-pixel motion vectors or not, -with the RANGE and PIXEL parameter file statements. - -
In MPEG-1, a movie is represented as a sequence of MPEG frames, -each of which is an I Frame, a P Frame, or a B Frame. Each represents -an actual frame of the movie (don't get confused by the dual use of -the word "frame." A movie frame is a graphical image. An MPEG frame -is a set of data that describes a movie frame). - -
An I frame ("intra" frame) describes a movie frame in isolation -- -without respect to any other frame in the movie. A P frame -("predictive" frame) describes a movie frame by describing how it -differs from the movie frame described by the latest preceding I or -P frame. A B frame ("bidirectional" frame) describes a movie frame by -describing how it differs from the movie frames described by the -nearest I or P frame before and after it. - -
Note that the first frame of a movie must be described by an I -frame (because there is no previous movie frame) and the last movie -frame must be described by an I or P frame (because there is no -subsequent movie frame). - -
Beyond that, you can choose which frames are represented by which -types. You specify a pattern, such as IBPBP and ppmtompeg -simply repeats it over and over throughout the movie. The pattern -affects speed, quality, and stream size. Here is a chart which shows -some of the trade-offs: - -
| Frame Type | -Size | -Speed | -Quality | -
|---|---|---|---|
| I frames | -1000 | -1000 | -1000 | -
| P frames | -409 | -609 | -969 | -
| B frames | -72 | -260 | -919 | -
A standard sequence is IBBPBBPBBPBBPBB. - -
Select the sequence with the PATTERN parameter file statement. - -
Since the last MPEG frame cannot be a B frame (see above), if the -pattern you specify indicates a B frame for the last movie frame of -the movie, ppmtompeg makes it an I frame instead. - -
Before Netpbm 10.26 (January 2005), ppmtompeg instead drops -the trailing B frames by default, and you need the -FORCE_ENCODE_LAST_FRAME parameter file statement to make it do -this. - -
The MPEG frames don't appear in the MPEG-1 stream in the same order that -the corresponding movie frames appear in the movie -- the B frames come after -the I and P frames on which they are based. For example, if the movie is -4 frames that you will represent with the pattern IBBP, the MPEG-1 stream -will start with an I frame describing movie frame 0. The next frame in -the MPEG-1 stream is a P frame describing movie frame 3. The last two -frames in the MPEG-1 stream are B frames describing movie frames 1 and 2, -respectively. - - -
Specify the input frame images with the INPUT_DIR, -INPUT, END_INPUT, BASE_FILE_FORMAT, -SIZE, YUV_FORMAT and INPUT_CONVERT parameter -file statements. - -
Specify the output file with the OUTPUT parameter file statement. - - -
ppmtompeg can generate a variety of statistics about the -encoding. See the -stat, -snr, -mv_histogram, --quiet, -no_frame_summary, and -bit_rate_info -options. - - -
You can run ppmtompeg on multiple machines at once, encoding -the same MPEG stream. When you do, the machines are used as shown in -the following diagram. We call this "parallel mode." - -
-
-
To do parallel processing, put the statement - -
- PARALLEL -- -in the parameter file, followed by a listing of the machines, one -machine per line, then - -
- END_PARALLEL -- -Each of the machine lines must be in one of two forms. If the machine -has filesystem access to the input files, then the line is: - -
-machine user executable - -
The executable is normally ppmtompeg (you may need to give -the complete path if you've built for different architectures). If -the machine does not have filesystem access to the input files, the line -is: - -
REMOTE machine user executable -parameter file - -
The -max_machines command option limits the number of -machines ppmtompeg will use. If you specify more machines in -the parameter file than -max_machines allows, ppmtompeg -uses only the machines listed first. This is handy if you want to -experiment with different amounts of parallelism. - -
In general, you should use full path file names when describing -executables and parameter files. This includes the parameter -file argument on the original invocation of ppmtompeg. - -
All file names must be the same on all systems (so if e.g. you're -using an NFS filesystem, you must make sure it is mounted at the same -mountpoint on all systems). - -
Because not all of the processes involved in parallel operation -have easy access to the input files, you must specify the SIZE -parameter file statement when you do parallel operation. - -
The machine on which you originally invoke ppmtompeg is the -master machine. It hosts a "combine server,", a -"decode server," and a number of "i/o servers," -all as separate processes. The other machines in the network (listed -in the parameter file) are slave machines. Each hosts a single -process that continuously requests work from the master and does it. -The slave process does the computation to encode MPEG frames. It -processes frames in batches identified by the master. - -
The master uses a remote shell command to start a process on a -slave machine. By default, it uses an rsh shell command to do -this. But use the RSH parameter file statement to control -this. The shell command the master executes remotely is -ppmtompeg, but with options to indicate that it is to perform -slave functions. - -
The various machines talk to each other over TCP connections. Each -machine finds and binds to a free TCP port number and tells its -partners the port number. These port numbers are at least 2048. - -
Use the PARALLEL_TEST_FRAMES, PARALLEL_TIME_CHUNKS, and -PARALLEL_PERFECT parameter file statements to control the way the -master divides up work among the slaves. - -
Use the -nice command option to cause all slave processes to run -"nicely," i.e. as low priority processes. That way, this substantial and -long-running CPU load will have minimal impact on other, possibly -interactive, users of the systems. - - -
Here is a look at ppmtompeg speed, in single-node (not parallel) -operation: - -
| Machine Type | -Macroblocks per second1 | -
|---|---|
| HP 9000/755 | -280 | -
| DEC 3000/400 | -247 | -
| HP 9000/750 | -191 | -
| Sparc 10 | -104 | -
| DEC 5000 | -68 | -
The measurements in the table are with inputs and outputs via a -conventional locally attached filesystem. If you are using a network -filesystem over a single 10 MB/s Ethernet, that constrains your speed more -than your CPU speed. In that case, don't expect to get better than 4 -or 5 frames per second no matter how fast your CPUs are. - -
Network speed is even more of a bottleneck when the slaves do not -have filesystem access to the input files -- i.e. you declare them -REMOTE. - -
Where I/O is the bottleneck, size of the input frames can make a big -difference. So YUV input is better than PPM, and JPEG is better than -both. - -
When you're first trying to get parallel mode working, be sure to -use the -debug_machines option so you can see what's going on. -Also, -debug_sockets can help you diagnose communication -problems. - - -