| Q. | What is it (not)? |
|---|---|
| A. | Maybe to your disappointment it is not about video. The scope of this page is computer storage applications, things like backup, archiving, data exchange... The downloadable files are an optional kernel patch and a couple of user-land utilities dubbed as dvd+rw-tools. |
| Q. | Kernel patch? What's wrong with cdrecord/cdrtools? |
|---|---|
| A. | Nothing! Moreover, you actually need them (cdrtools).
It's just that you can't use them to manage the DVD+RW nor
DVD+R media itself. DVD+RW media has no notion of [multiple]
sessions or packet writing, so cdrecord[-ProDVD]
wouldn't really know what to do (nor would dvdrtools
despite what RedHat
7.3 Release Notes say). But most important is that DVD+RW
is a true random write access media and therefore is suitable
for housing of arbitrary filesystem, e.g. udf, vfat, ext2, etc.
This, and this alone, qualifies DVD+RW support for kernel
implementation. However! Support for arbitrary
filesystem doesn't seem to work with 1st generation drives,
Ricoh MP5120A and derivatives (see Technical Ramblings below
for further details).
It should be explicitly noted that dvd+rw-tools do not require kernel support and if they fulfill your expectations, then the patch is by all means optional. |
| Q. | What are dvd+rw-tools for? |
|---|---|
| A. | To format blank DVD+RW media to start with. Secondly, even though you [might] have an opportunity to put for example an ext2 filesystem on DVD+RW, it's not very practical because you most likely want to access the data on arbitrary computer. Or in other words you most likely want ISO9660. Trouble is that you very likely want to add data now and then. And what options do you have in the lack of multiple sessions? Complete remastering which takes more and more time as dataset grows? Well, yes, unless you employ growisofs! growisofs provides the way to lay down and grow an ISO9660 filesystem on, as well as to burn an arbitrary pre-mastered image to both DVD+RW and DVD+R media. |
| Q. | [grow]isofs? Isn't UDF the one to use? |
|---|---|
| A. | My initial experience [from late 2001] with UDF was rather poor. I couldn't even mount an UDF volume created under Windows [with Veritas DLA] in either Linux or Solaris. Now it doesn't appear to be an issue anymore... Your milage may of course vary... But one way or another I still have to recommend to deploy it with caution, see tutorial for further details. |
| Q. | What do I need cdrtools for? |
|---|---|
| A. | The DVD+RW drives available on the market can burn even CD-R[W] media and cdrecord is the tool for this job [and this job only]. Besides, growisofs is just a front-end [rather back-end] to mkisofs. |
Provided the absolute lack of support from vendor(s) you should consider this page to be a result of guesswork. There still is a couple of fatal failures indicated by vendor-specific error codes (see Technical Ramblings) and I didn't yet figured out the way around 'em. So the status of this "project" is rather "gathering experience" than "ready for production environment."
If you have an IDE unit, make sure it is "routed" through ide-scsi emulation layer by either:
options ide-cd ignore=hdX pre-install sg modprobe ide-scsi pre-install sr_mod modprobe ide-scsi pre-install ide-scsi modprobe ide-cd
Keep in mind that once hdX is routed through ide-scsi, you can no longer refer to /dev/hdX, but to corresponding /dev/scdN only.
If you have an external unit, just get it working as CD-ROM first. I myself have no personal experience whatsoever with USB or IEEE1394/Firewire storage devices of any kind and have to direct you elsewhere for specific instructions. I however am confident that if you manage to get your drive working reliably as CD-ROM and CD-R[W] burner, then you won't have any troubles with DVD+RW part. As for the moment of this writing, USB connected drives were reported to be working fine. Firewire connected drives in turn were reported to fail miserably under 2.4.18. The failure didn't seem to be DVD+RW related as it reportedly failed burning even CD-R media. Firewire support was substantially revamped in 2.4.19, and dvd+rw-tools were reported to work with this kernel.
If you're running 2.4.19, consider applying this patch. I write "consider" and not "do" for following reasons:
If you have 1st generation drive (Ricoh MP5120A and derivatives) do consider upgrading your firmware (see Ricoh, HP, MET pages). Fixed bug descriptions are vague, but at the very least after upgrade to 1.34 I could no longer reproduce infrequent "random positioning errors" when reading a file system with a lot of small files. 1.34 adds support for Verbatim media and 1.37 apparently for Memorex. Version 1.37 also implements a secret vendor-specific command which reportedly improves compatibility with a number of DVD-ROM drives (more about this matter in Technical Ramblings).
Download, unpack and compile the the toolchain. Note that separate source code files found in the download catalog are provided mainly for references purposes. To build the thing do choose the .tar.gz archive which contains Makefile as well as .spec file.
Formatting the DVD+RW media. Just pass the device name, e.g. /dev/scd0, as an argument to dvd+rw-format. To make formatting process reasonably fast the media gets formatted only partially, as you can notice by observing a progress indicator displayed by the program. The final indicator value varies from firmware to firmware, values as low as 1.6% were observed. But it does not mean that you can only write that little. The unit keeps formatting transparently, as you add more data.
It was observed that excessive reformats can render media unusable already after 10-20 reformats and [enforced] reformat is therefore not recommended. As you might remember former recommendatations for reformatting were for privacy reason. Version 2.x is not suitable for this purpose, because so called "immediate" form (first utilized in version 2.0) of the "FORMAT UNIT" command does not attempt to zero all the data written so far. If you are concerned about privacy, nullify explicitely with 'growisofs -Z /dev/scdN=/dev/zero' (or 'dd if=/dev/zero of=/dev/scdN bs=32k' if you've patched the kernel and like dd better:-).
DVD+R media does not require any formatting procedure applied and is ready to use out-of-the-box. Apparently, a reminder that 1st generation units (Ricoh MP5120A and derivatives) are not capable of burning DVD+R is needed.
Using growisofs. There is no manual for growisofs as there is very little need for one. In a nutshell growisofs just passes all command line arguments to mkisofs and dumps its output directly onto the media. The first part means that you basically can [well, should] consult mkisofs manual page and accompanying reference documention (including multisession related section[s]) and the second part means that you shouldn't expect an ISO-image on the standard output (nor make sure you have enough free temporary storage:-). Differences from mkisofs command line are:
Otherwise everything that applies to [multisession] mastering with mkisofs applies to growisofs as well. For example just like with mkisofs you should make a note on which options you used to master the initial "session" with and stick to them, e.g.:
growisofs -Z /dev/scd0 -R -J /some/files growisofs -M /dev/scd0 -R -J /other/files
Oh! Do make sure you have at least mkisofs 1.14 on your $PATH (mkisofs 1.14 is part of cdrtools 1.10).
In addition, growisofs [version 3.3 or later] recognizes special form of -Z command line option which permits burning of arbitrary pre-mastered images. The "magic" command is:
growisofs -Z /dev/scd0=image.iso
where image.iso represents an arbitrary object in the filesystem, such as file, named pipe or device entry. No, nothing is growing here and command name is counter-intuitive in this particular context. And even less intuitive:-) If you want to burn output generated by an arbitrary program, you can use:
dumpsomething | growisofs -Z /dev/scd0=/dev/fd/0
Burning DVD+R implies extra limitations:
If you're satisfied with growisofs, then patching the kernel is optional. If you haven't stopped reading beyond this line, download the patch, apply it, rebuild the kernel or modules and re-install (kernel or cdrom.o and sr_mod.o modules, whichever appropriate), but don't ask me how. To see it in action, insert formatted DVD+RW media and try to access it, 'dd if=/dev/scdN count=0' would do. Then verify that kernel logs "srN: mmc-3 profile: 1Ah". You should now be able to 'mkisofs -pad . | dd of=/dev/scdN obs=32k' or even 'mke2fs -b 2048 /dev/scdN' and observe kernel logging "srN: dirty DVD+RW media."
If you have previous patch version applied, then you have to back it out first. The simplest way is probably to restore drivers/scsi/sr*.[ch] and drivers/cdrom/cdrom.c from your original Linux source code ditribution.
Even though kernel now permits to build and mount arbitrary filesystem, there is one thing you must keep in mind before you just proceed, no matter how tempting it might appear.
As you might know DVD+RW media can sustain only around 1000 overwrites. Now the thing about fully fledged filesystems is that every read [or tight bunch of 'em] is accompanied by corresponding i-node update or in other words a write! Now let's say you lookup the mount point (e.g. ls /mnt/dvd) ten times a day. This gives you a 100 days lifetime on your mountpoint and therefore media. Not really much, huh? So do use noatime mount option with DVD+RW media or have it mounted read-only most of the time. However! Every read-write mount "costs" a super-block update. So that if you remount the media say 3 times a day, it would last for about a year [supermount would exhaust the "budget" way sooner]... Defect management [in firmware, a.k.a. Mt.Rainier, or at filesystem level] would improve the situation, but ideally filesystem driver should definitely refrain from modifying the superblock [marking it dirty] if nothing was actually written since last mount. Given the development status of Linux UDF the chances for seeing the latter implemented [for UDF] are more than just conceivable. The request is already filed and even possible solution is being discussed. But why not give UDF a shot already then? By default UDF write support is unfortunately disabled and you might have to reconfigure the kernel and rebuild modules. Alternatively [my preferred option actually] fetch the code at SourceForge and build the module separately. Of course you will have to fetch and build udftools as well. But once it's done just type:
mkudffs --spartable=2 --media-type=cdrw /dev/scdN mount -o rw,noatime /dev/scdN /mnt/cdrom
mkudffs command line options were suggested by UDF maintainer, Ben Fennema.
Performance optimizations. This paragraph applies only if you've patched the kernel. As some of you might remember the original recommendation was "do use obs=32k for optimal performance." Well, it was rather naive of me to say so, as common block device layer completely reorganizes the stream so that '>/dev/scd0' is as good as '|dd of=/dev/scd0 obs=32k'. It should also be noted that dumping to /dev/scd0 puts quite a pressure on VM subsystem, as the data passes through block buffer cache. To minimize the pressure and improve overall system performance bind the cdrom device to a raw device, e.g. 'raw /dev/raw/raw1 /dev/scd0', growisofs will locate and use it automatically. obs=32k makes perfect sense with /dev/raw devices, but dd won't work as /dev/raw expects specifically aligned buffer... Good news are that sdd aligns the transfer buffer so that if you've got an image to burn down (or want to nullify) stick to 'sdd of=/dev/raw/rawX obs=32k ...'
In order to optimize seek times [most?] DVD[-ROM] players calibrate their mechanics at the moment of media mount by sliding the optical head some place, picking up the signal and noting the physical block address underneath the lens. In order for this procedure to work with rewritable/recordable media, that particular spot has to be written to [or de-iced in DVD+RW case]. Some units slide the head to 30mm [radial] to calibrate. In order to keep such players "happy," make sure that at least 500MB is written.
Other units attempt to seek to lead-out [or vicinity of it] for calibration purposes. Now the catch is that it's perfectly possible to produce a DVD+RW disk without lead-out. Most notably media initially formatted with dvd+rw-format [apparently] doesn't have any lead-out, not to mention that practically whole surface remains virgin. If you fail to mount/play DVD+RW media, attempt to
dvd+rw-format -lead-out /dev/scdN
which relocates the lead-out next to outermost written sector as well as makes sure there is no virgin surface before it.
Then non-finalized DVD+R disks don't have lead-out either(*). If you fail to mount/play DVD+R media and wish to sacrifice the remaining space for immediate compatibility, just fill the media up(**). Alternatively if you master DVD+R disc in a single take and don't plan to use it for multi-sessioning(***), you have the option to invoke growisofs with -dvd-compat option and cut the real lead-out already at the end of first session.
| (*) | Well, there are lead-outs at the session edges, but the problem is that "End Physical Sector Number of Data Area" field in "Control Data Zone" of the lead-in contains address of the largest media sector, which makes DVD[-ROM] players look for lead-out at the outermost edge instead of the first session. Actually I fail to understand why don't they burn the address of last sector of the first session in the lead-in even on multi-session disks? Something to correct in next firmware update? |
| (**) | The easiest way is to create couple of holey files with 'touch hugeN.void' and 'perl -e 'truncate ("hugeN.void", 0x7ffffffe)'', and finally to 'growisofs ... huge*.void'. The command is bound to fail with "end of medium reached," but that's what you basically want. |
| (***) | E.g. when mastering DVD-Video disk:-) Note that -dvd-video option [passed to mkisofs] engages -dvd-compat automatically. |
Then we have logical format compatibility issue(s). The very ground for all the controversy around DVD+RW, rather around DVD+RW media not being playable in a whole range of players. DVD+RW Alliance was keen to blame on DVD-ROM vendors, even claiming that they deliberately block playback. But the fact is that format specifications don't explicitely say that unrecognized format [designated by "Book Type" field in "Control Data Zone" of the lead-in] should be treated as DVD-ROM and [in my opinion] it was rather naive of them to claim and expect that the media will be playable in "virtually all players." This deficiency was recognized by practically all DVD+RW vendors [apparently all except Sony] and a secret vendor-specific command manipulating this "Book Type" field was implemented. If you fail to mount/play DVD+RW media, attempt to
dvd+rw-booktype -dvd-rom -media /dev/scdN
It's naturally not possible to manipulate the "Book Type" field on DVD+R media, that is not after the lead-in is written [which takes place at the moment the first session gets closed]. But it's possible to control how it [lead-in] is going to be branded by programming the drive in advance:
dvd+rw-booktype -dvd-rom -unit+r /dev/scdN
Meaning that if you fail to play DVD+R media, you can attempt to burn another disc with different unit settings. For more background information about dvd+rw-booktype, see "Compatibility Bitsettings" article at dvdplusrw.org.
There [potentially] are other logical format incompatibilites, but the "Book Type" issue discussed above is the only one "officially" recognized. Well, it's actually understandable as it's the only one that can be recognized and addressed by a DVD+RW vendor alone. Recognition of other incompatibilities would require cooperation from DVD[-ROM] player vendors and that's something they apparently are not willing to show referring to the fact that DVD+RW format is not approved [and apprently never will be] by DVD Forum(*).
| (*) | To which I say "so what?" DVD Forum is an alliance of manufacturers just like DVD+RW Alliance is. It [or any other party for that matter] has no authority to deny a technology development initiative. |
Finally there is a physical incompatibility issue. They claim that there're optical pick-ups out there not being capable to decode the track because of low reflectivity [of DVD+RW media surface]. I write "they claim," because in the lack of cooperation from DVD[-ROM] vendors it's not possible to distinguish physical from logical format incompatibility, which I find important to tell apart in order to make sure at least logical format incompatibility issues don't persist over time. But either way, there is very little you can do about this one, but to try another player...
What does "DVD+RW support" really mean? Even though DVD+RW has no notion of [multiple] sessions, to ensure compatibility with DVD-ROM it's essential to issue "CLOSE TRACK/SESSION (5Bh)" MMC command to terminate/suspend background formatting (if any in progress) whenever you intend to eject the media or simply stop writing and want better read performance (e.g. remount filesystem read-only). This is what the patch is basically about: noting when/if media was written to and "finalizing" at unlock door.
Secondly, whenever you employ fully fledged file system, I/O requests get inevitably fragmented. "Fragmented" means following. Even though you can address the data with 2KB granilarity, it [data] is physically laid out in 32KB chunks. This in turn means that for example writing of 2KB block involves reading of 32KB chunk, replacing corresponding 2KB and writing down of modified 32KB chunk. "Fragmented requests" are those that are smaller than 32KB or/and cross the modulus 32KB boundaries. In order to optimize the process certain caching algorithm is implemented in unit's firmware. Obviously it can't adequately meet all possible situations. And so in such unfortunate situations the drive apparently stops processing I/O requests returning "COMMAND SEQUENSE ERROR (2Ch)" ASC. This is the second essential of "DVD+RW support," namely injecting of "SYNCHRONIZE CACHE (35h)" MMC command in reply to the error condition in question. The command flushes the cached buffers which makes it possible to resume the data flow.
Unfortunately the above paragraph doesn't seem to apply to the 1st generation drives, Ricoh MP5120A and derivatives:-( "SYNCHRONIZE CACHE (35h)" doesn't seem to be sufficient and the unit keeps replying with "COMMAND SEQUENSE ERROR (2Ch)" going into end-less loop. This makes it impossible to deploy arbitrary file system. I'm open for suggestions... Meanwhile the I've chosen to simply suspend I/O till the media is unmounted.
Even 2nd gen unit were reported to exhibit similar [but not the same] behaviour under apparently extremely rare circumstanses. At least I failed to reproduce the problem... But it's being looked into...
This one really beats me. Sometimes the unit simply stops writing signalling a vendor specific positioning error, 03h/15h/82h to be specific. Especially if the media is newly formatted. Couple of work theories. One is that block buffer cache reorders requests so that they are not sequential anymore, "FLUSH CACHE" might do the trick. Another one is that under "underrun" condition background formatting kicks off and has to be explicitely stopped. "Underrun" is in quotes because the unit is supposed to handle temporary data stream outages gracefully. If you run into this (you most likely will), try to complement growisofs command line with [undocumented] -poor-man option (which has to be first in the command line). This option eliminates request reorders and minimizes possibility for "underrun" condition (by releasing the pressure off VM subsystem).
The original idea was to implement DVD+RW support in drivers/cdrom/cdrom.c. Unfortunately SCSI layer maintains private "writeable" flag controlling the ability to issue WRITE commands. The flag is impossible to reach for from the Unified CD-ROM driver. But why am I talking about SCSI when there are only IDE units out there (at least for the time being)? Well, as you most likely want to occasionally burn even CD-R[W] with cdrecord you want it to go through ide-scsi emulation layer anyway. So I figured that SCSI CD-ROM driver is the one to aim for even for DVD+RW.
Unfortunately it was not possible to implement it completely in sr_mod.o:-( Minor drivers/cdrom/cdrom.c modification was required to sense the media before decision about whether or not to permit write open. That's because DVD+RW drives are morphing their behaviour after currently mounted media and it's essential to identify newly inserted media.
Special comment about "what a dirty hack!!!" To my great surprise it turned out that timeout value you pass in cdrom_generic_command is simply ignored and timeout is set to precompiled value of 30 seconds. Unfortunately it's way too low for formatting purposes and I had to do something about it. Alternative to "the dirty hack" was to add another argument to sr_do_ioct and modify all the calls to it... I've chosen to take over those 31 unused bits from the "quiet" argument instead of modifying all the calls (too boring).
There is another kernel "deficiency" I ran into while working on the (original version of) dvd+rw-format utility. The drive provides background formatting progress status, but unfortunately it's impossible to access it. That's because progress counter is returned [in reply to "TEST UNIT READY"] as "NO SENSE/LOGICAL UNIT NOT READY/FORMAT IN PROGRESS" sense bytes but with "GOOD" status. Apparently any sense data with "GOOD" status is discarded by the common SCSI layer.
It was pointed out to me that DVD+RW units work with Acard SCSI to IDE bridges.