404
Page not found :(
The requested page could not be found.
File APIs
Mac OS filesystem APIs evolved as the underlying filesystem semantics changed.
Pascal Strings
Prior to Mac OS X, Macintosh APIs used Pascal strings to represent strings. Pascal strings are passed by pointer. The first byte pointed to stores the string length, and the string contents follows. Pascal strings are not null-terminated and not compatible with C-style strings. Note that since the string length is stored in a single byte, these strings cannot be longer than 255 bytes.
Compilers for Mac OS support Pascal strings by putting the \p sequence at the beginning of a string. For example,
const unsigned char kFilename[] = "\pMy File";
+ File APIs - SyncFiles Documentation File APIs | SyncFiles Documentation File APIs
Mac OS filesystem APIs evolved as the underlying filesystem semantics changed.
Pascal Strings
Prior to Mac OS X, Macintosh APIs used Pascal strings to represent strings. Pascal strings are passed by pointer. The first byte pointed to stores the string length, and the string contents follows. Pascal strings are not null-terminated and not compatible with C-style strings. Note that since the string length is stored in a single byte, these strings cannot be longer than 255 bytes.
Compilers for Mac OS support Pascal strings by putting the \p sequence at the beginning of a string. For example,
const unsigned char kFilename[] = "\pMy File";
This is equivalent to:
const unsigned char kFilename[8] = {
7, 'M', 'y', ' ', 'F', 'i', 'l', 'e'
};
@@ -25,7 +25,7 @@
const unsigned char * fileName,
OSType creator,
OSType fileType);
-
In this API, files are identified by volume ID, the directory ID within that volume, and the filename within that directory. The encoding is not specified, and presumably, files will be created using the system’s default character encoding.
FSSpec API
The FSSpec API does not change semantics, but provides a simple data structure which is used to store the volume ID, directory ID, and filename. This structure is called FSSpec.
struct FSSpec {
+
In this API, files are identified by volume ID, the directory ID within that volume, and the filename within that directory. The encoding is not specified, and presumably, files will be created using the system’s default character encoding.
FSSpec API
Mac System 7 introduces the FSSPec API. It does not change semantics, but provides a simple data structure which is used to store the volume ID, directory ID, and filename. This structure is called FSSpec.
struct FSSpec {
short vRefNum;
long parID;
unsigned char name[64];
diff --git a/tech/filesystems.html b/tech/filesystems.html
index 0c069bd..6f212f9 100644
--- a/tech/filesystems.html
+++ b/tech/filesystems.html
@@ -1 +1 @@
- Filesystems - SyncFiles Documentation Filesystems | SyncFiles Documentation Filesystems
This document focuses on the different filesystems used by Mac OS over the years and how filenames work on these filesystems.
MFS
Macintosh file system (MFS) is Apple’s filesystem for the first Macintosh. MFS does not support directories and has a maximum filename length of 63 characters. According to Wikipedia, the final OS versions that supported MFS were 7.6 for read-write access and 8.0 for read-only access.
It is unusual to see this filesystem in practice, since it was replaced by HFS shortly after it appeared.
HFS
Hierarchical file system (HFS) was introduced shortly after MFS and replaced MFS. It first appeared alongside Apple’s first Macintosh hard disk, the “Hard Disk 20” in 1985, and afterwards appeared in the 128K ROM on the Mac Plus. HFS introduces directories and a new set of file APIs.
The final versions of Mac OS which support HFS are Mac OS X 10.5 Leopard for read-write access and macOS 10.14 Mojave for read-only access.
HFS has a maximum filename length to 31 characters. The script used for the filename is recorded, but filenames are compared as if they are were encoded using the Macintosh Roman encoding. Filenames are case insensitive, and the sort order is described on page A-20 of Inside Macintosh: Text (1993).
HFS Plus
HFS Plus was introduced with Mac OS 8.1 and added support for filesystem journaling, Unicode filenames, up to 255 characters per filename, and case-sensitive filenames if enabled. Mac OS 8.1 was released in January 1998.
To provide backwards compatibility with older APIs, HFS Plus records the encoding that filenames should be encoded with when listing files in the older APIs. The volume header tracks a list of all encodings used for all filenames on the volume, so the appropriate conversion tables can be loaded when the volume is mounted. When an application using an older API lists the files in a directory, it will see backwards-compatible filenames substituted for filenames that use unsupported characters or filenames that are too long.
Filenames are stored in UTF-16, decomposed using the rules from Unicode 2.1 (up to Mac OS X 10.2) or Unicode 3.2 (for Mac OS X 10.3 and later).
See [Apple Technical Note TN1150: HFS Plus Volume Format][tn1150].
There is a variant of HFS Plus called HFSX. The major difference between normal HFS Plus and HFSX is that HFSX does not carry an HFS wrapper for backwards compatibility with systems that do not support HFS Plus.
APFS
APFS is introduced in macOS 10.12.4. Filenames are encoded using UTF-8. Only code points assigned in Unicode 9.0 are permitted in filenames. APFS does not normalize filenames, but does store files by using the hash of the normalized version of the filename.
UFS
UFS stands for Unix file system. It is a case-sensitive filesystem which is only supported by Mac OS X versions 10.0 through 10.5.
UFS is not seen often.
Disk Images
There are various formats for disk images: Disc Copy 4.2 images, NDIF images, and UDIF images.
+ Filesystems - SyncFiles Documentation Filesystems | SyncFiles Documentation Filesystems
This document focuses on the different filesystems used by Mac OS over the years and how filenames work on these filesystems.
MFS
Macintosh file system (MFS) is Apple’s filesystem for the first Macintosh. MFS does not support directories and has a maximum filename length of 63 characters. According to Wikipedia, the final OS versions that supported MFS were 7.6 for read-write access and 8.0 for read-only access.
It is unusual to see this filesystem in practice, since it was replaced by HFS shortly after it appeared.
HFS
Hierarchical file system (HFS) was introduced shortly after MFS and replaced MFS. It first appeared alongside Apple’s first Macintosh hard disk, the “Hard Disk 20” in 1985, and afterwards appeared in the 128K ROM on the Mac Plus. HFS introduces directories and a new set of file APIs.
The final versions of Mac OS which support HFS are Mac OS X 10.5 Leopard for read-write access and macOS 10.14 Mojave for read-only access.
HFS has a maximum filename length to 31 characters. The script used for the filename is recorded, but filenames are compared as if they are were encoded using the Macintosh Roman encoding. Filenames are case insensitive, and the sort order is described on page A-20 of Inside Macintosh: Text (1993).
HFS Plus
HFS Plus was introduced with Mac OS 8.1 and added support for filesystem journaling, Unicode filenames, up to 255 characters per filename, and case-sensitive filenames if enabled. Mac OS 8.1 was released in January 1998.
To provide backwards compatibility with older APIs, HFS Plus records the encoding that filenames should be encoded with when listing files in the older APIs. The volume header tracks a list of all encodings used for all filenames on the volume, so the appropriate conversion tables can be loaded when the volume is mounted. When an application using an older API lists the files in a directory, it will see backwards-compatible filenames substituted for filenames that use unsupported characters or filenames that are too long.
Filenames are stored in UTF-16, decomposed using the rules from Unicode 2.1 (up to Mac OS X 10.2) or Unicode 3.2 (for Mac OS X 10.3 and later).
See Apple Technical Note TN1150: HFS Plus Volume Format.
There is a variant of HFS Plus called HFSX. The major difference between normal HFS Plus and HFSX is that HFSX does not carry an HFS wrapper for backwards compatibility with systems that do not support HFS Plus.
APFS
APFS is introduced in macOS 10.12.4. Filenames are encoded using UTF-8. Only code points assigned in Unicode 9.0 are permitted in filenames. APFS does not normalize filenames, but does store files by using the hash of the normalized version of the filename.
UFS
UFS stands for Unix file system. It is a case-sensitive filesystem which is only supported by Mac OS X versions 10.0 through 10.5.
UFS is not seen often.
Disk Images
There are various formats for disk images: Disc Copy 4.2 images, NDIF images, and UDIF images.
diff --git a/tech/finder-info.html b/tech/finder-info.html
index 5740e1e..26db5a7 100644
--- a/tech/finder-info.html
+++ b/tech/finder-info.html
@@ -1,4 +1,4 @@
- Finder Info - SyncFiles Documentation Finder Info | SyncFiles Documentation Finder Info
Mac OS also stores a small piece of metadata for each file called Finder Info. As the name implies, this contains information which is primarily used by the Finder (Finder is the Mac OS shell, which shows the desktop and filesystem). For example, this includes the file’s type and its location on-screen.
There are different versions of the Finder info structure. This is what one of the older versions looks like, for a file:
struct FileInfo {
+ Finder Info - SyncFiles Documentation Finder Info | SyncFiles Documentation Finder Info
Mac OS also stores a small piece of metadata for each file called Finder Info. As the name implies, this contains information which is primarily used by the Finder (Finder is the Mac OS shell, which shows the desktop and filesystem). For example, this includes the file’s type and its location on-screen.
There are different versions of the Finder info structure. This is what one of the older versions looks like, for a file:
struct FileInfo {
OSType fileType;
OSType fileCreator;
UInt16 finderFlags;
diff --git a/tech/index.html b/tech/index.html
index 55f1783..26ea46b 100644
--- a/tech/index.html
+++ b/tech/index.html
@@ -1 +1 @@
- Technical Guide - SyncFiles Documentation Technical Guide | SyncFiles Documentation
+ Technical Guide - SyncFiles Documentation Technical Guide | SyncFiles Documentation
diff --git a/tech/resource-forks.html b/tech/resource-forks.html
index a9e6153..c0b13a5 100644
--- a/tech/resource-forks.html
+++ b/tech/resource-forks.html
@@ -1 +1 @@
- Resource Forks - SyncFiles Documentation Resource Forks | SyncFiles Documentation Resource Forks
Mac OS traditionally provided two forks for a file: the data fork and the resource fork. A fork is a data stream within a file which can be independently manipulated. You can open one fork and write data to it, change the length of data, or delete the fork entirely without affecting the file’s other fork. Think of the forks as two separate files, bundled up as one file.
When you copy a file from a non-Mac OS system to a Mac, what you get is a file that only contains a data fork. The resource fork is used to store Mac-specific data. It’s almost always organized into chunks called resources, which are discrete pieces of data surch as icons, images, sounds, strings, or 68K code segments. Each resource is identified by a four-character type code and 16-bit ID number.
The resource fork format has a maximum size of about 16 MiB because it encodes file offsets using 24 bits.
Historical Use
Prior to Mac OS X, programs on the Mac make heavy use of the resource fork. For example, an application’s resource fork contains 68K code, icons, dialog box layouts, version information, text data, and sometimes various custom data types. PowerPC code is stored in the data fork, although various resources are still necessary for PowerPC applications to run correctly.
Applications also use resource forks in other files to store data. Some text editors use the resource fork of a text file to remember the state of the text editor when editing that file. Games often use files with resource forks to store images, sound effects, or level data.
Starting with Mac OS X, data that was previously stored as resources in the resource fork are stored as a separate file instead. For example, application and file icons prior to Mac OS X are stored in the resource fork, but in Mac OS X, each icon is stored as a separate file. It is unusual to find a Mac OS X program that uses the resource fork at all.
Are Resource Forks Really Structured?
Note the key words above: a resource fork is almost always organized into chunks called resources.
A file’s resource fork is really just an alternate stream of data, like the data fork. You can put whatever data you like in the resource fork. In practice, the resource fork almost always uses a specific format.
Note that this goes both ways. Just like you can store arbitrary data in the resource fork, you can also use the data fork to store resources. Using the data fork to store resources has disadvantages, because you can’t edit those resources with ResEdit or use the Macintosh resource manager API to read those resources, so it is rarely done.
Some applications store application preferences or other data in the resource fork of a file, and because resource forks can get corrupted, you occasionally see a backup copy of the resource fork stored in the data fork of the same file. This is not common, however.
Preserving Resource Forks
Resource forks do not always need to be preserved when synchronizing files between systems. It depends on the file and what is being stored in the resource fork. For example, when you save a text file in BBEdit or MPW, the editor state is recorded as a resource in the text file. Deleting this resource fork doesn’t affect your ability to use the file. On the other hand, if you delete the resource fork of an application, it won’t work at all.
Examining the Resource Fork
You can use ResEdit or Resorcerer to view and edit the resources in a resource fork, or the MPW tools Rez and DeRez.
ResEdit is the most common tool to use, because it’s free (unlike Resorcerer, which costs $256) and has a nice user interface (unlike Rez and DeRez, which convert resource files to and from text files). ResEdit is available from Apple and the latest version is 2.1.3. Here’s what the resource fork of the MacBinary application looks like in ResEdit:
When you open a file in ResEdit, ResEdit shows you an overview of the different types of resources in the file. MacBinary contains 18 different types of resources. When you open one of the types, ResEdit shows you a list of resources for that type. For example, we can look at the CODE resources in MacBinary, which contain segments of executable 68K code which can be independently loaded. Each individual resource has an ID, which is a signed 16-bit number, and optionally has a name.
ResEdit provides simple editors for certain types of resources, like icons. This makes it easy to create your own custom icons for Macintosh applications. You didn’t have to be a programmer to take advantage of this—if you wanted to create your own custom folder icons for certain folders, you could do that too. You could find custom icon collections online or on CDs bundled with magazines.
Application and folder icons are made from several types of icons with the same ID number, creating an icon family. An icon family allows you to create custom icons for different sizes and color depths.
Another common type of resource is string resources. Applications sometimes store strings used by the application in these resources. This might be done to reduce the memory footprint of the application, or it might be done to make it easier to translate an application into another language.
It’s not necessary to use string resources at all, and MacBinary doesn’t contain any string resources. There are string resources in SimpleText, however, containing help messages, error messages, and various other strings.
+ Resource Forks - SyncFiles Documentation Resource Forks | SyncFiles Documentation Resource Forks
Mac OS traditionally provided two forks for a file: the data fork and the resource fork. A fork is a data stream within a file which can be independently manipulated. You can open one fork and write data to it, change the length of data, or delete the fork entirely without affecting the file’s other fork. Think of the forks as two separate files, bundled up as one file.
When you copy a file from a non-Mac OS system to a Mac, what you get is a file that only contains a data fork. The resource fork is used to store Mac-specific data. It’s almost always organized into chunks called resources, which are discrete pieces of data surch as icons, images, sounds, strings, or 68K code segments. Each resource is identified by a four-character type code and 16-bit ID number.
The resource fork format has a maximum size of about 16 MiB because it encodes file offsets using 24 bits.
Historical Use
Prior to Mac OS X, programs on the Mac make heavy use of the resource fork. For example, an application’s resource fork contains 68K code, icons, dialog box layouts, version information, text data, and sometimes various custom data types. PowerPC code is stored in the data fork, although various resources are still necessary for PowerPC applications to run correctly.
Applications also use resource forks in other files to store data. Some text editors use the resource fork of a text file to remember the state of the text editor when editing that file. Games often use files with resource forks to store images, sound effects, or level data.
Starting with Mac OS X, data that was previously stored as resources in the resource fork are stored as a separate file instead. For example, application and file icons prior to Mac OS X are stored in the resource fork, but in Mac OS X, each icon is stored as a separate file. It is unusual to find a Mac OS X program that uses the resource fork at all.
Are Resource Forks Really Structured?
Note the key words above: a resource fork is almost always organized into chunks called resources.
A file’s resource fork is really just an alternate stream of data, like the data fork. You can put whatever data you like in the resource fork. In practice, the resource fork almost always uses a specific format.
Note that this goes both ways. Just like you can store arbitrary data in the resource fork, you can also use the data fork to store resources. Using the data fork to store resources has disadvantages, because you can’t edit those resources with ResEdit or use the Macintosh resource manager API to read those resources, so it is rarely done.
Some applications store application preferences or other data in the resource fork of a file, and because resource forks can get corrupted, you occasionally see a backup copy of the resource fork stored in the data fork of the same file. This is not common, however.
Preserving Resource Forks
Resource forks do not always need to be preserved when synchronizing files between systems. It depends on the file and what is being stored in the resource fork. For example, when you save a text file in BBEdit or MPW, the editor state is recorded as a resource in the text file. Deleting this resource fork doesn’t affect your ability to use the file. On the other hand, if you delete the resource fork of an application, it won’t work at all.
Examining the Resource Fork
You can use ResEdit or Resorcerer to view and edit the resources in a resource fork, or the MPW tools Rez and DeRez.
ResEdit is the most common tool to use, because it’s free (unlike Resorcerer, which costs $256) and has a nice user interface (unlike Rez and DeRez, which convert resource files to and from text files). ResEdit is available from Apple and the latest version is 2.1.3. Here’s what the resource fork of the MacBinary application looks like in ResEdit:
When you open a file in ResEdit, ResEdit shows you an overview of the different types of resources in the file. MacBinary contains 18 different types of resources. When you open one of the types, ResEdit shows you a list of resources for that type. For example, we can look at the CODE resources in MacBinary, which contain segments of executable 68K code which can be independently loaded. Each individual resource has an ID, which is a signed 16-bit number, and optionally has a name.
ResEdit provides simple editors for certain types of resources, like icons. This makes it easy to create your own custom icons for Macintosh applications. You didn’t have to be a programmer to take advantage of this—if you wanted to create your own custom folder icons for certain folders, you could do that too. You could find custom icon collections online or on CDs bundled with magazines.
Application and folder icons are made from several types of icons with the same ID number, creating an icon family. An icon family allows you to create custom icons for different sizes and color depths.
Another common type of resource is string resources. Applications sometimes store strings used by the application in these resources. This might be done to reduce the memory footprint of the application, or it might be done to make it easier to translate an application into another language.
It’s not necessary to use string resources at all, and MacBinary doesn’t contain any string resources. There are string resources in SimpleText, however, containing help messages, error messages, and various other strings.
diff --git a/tech/safe-saving.html b/tech/safe-saving.html
new file mode 100644
index 0000000..5373ca8
--- /dev/null
+++ b/tech/safe-saving.html
@@ -0,0 +1,14 @@
+ Safe Saving - SyncFiles Documentation Safe Saving | SyncFiles Documentation Safe Saving
There are a number of different goals for when your program saves a file:
-
I/O errors should be reported. If the data does not make it to disk, then tell the user that the operation failed.
-
Saves are atomic. After saving, you either get the old version of the file or the complete new version of the file. If your program crashes, it’s okay if the old file is untouched, but it’s not okay if it’s been partially overwritten.
-
Saves preserve file references. Any references to a document (aliases or bookmarks) remain valid after modifying the document.
-
Saves do not change the creation date, or other metadata associated with the file.
If your first thought is, “that sounds like it could be complicated”, then you’re in good company. Theodore Ts’o wrote an article in 2009, Don’t fear the fsync! which covers some of these issues on Linux in detail.
Classic Mac OS
HFS and HFS+ support an operation which exchanges the contents of files. The high-level API call looks like this:
OSErr FSpExchangeFiles(
+ const FSSpec * source,
+ const FSSpec * dest);
+
This function exchanges the contents of the two files (both forks), and exchanges the modification dates, but leaves the other metadata alone.
The recipe for safe saving on HFS volumes is:
-
Save the document to a temporary file on the same volume.
-
Exchange the contents of the original file and the temporary file with FSpExchangeFiles.
You can test that FSpExchangeFiles is supported by a volume by getting the volume parameters. Not all filesystems support this operation.
Mac OS X
Mac OS X provides a Unix system call that provides the same functionality as FSpExchangeFiles, but with a Unix API.
int exchangedata(
+ const char * path1
+ const char * path2,
+ unsigned int options);
+
However, this function does not work on APFS.
Mac OS X 10.6+
Starting on Mac OS X 10.6, the Foundation framework provides a method for safely replacing an item on the filesystem with a new item. This method is present on NSFileManager:
- (BOOL)replaceItemAtURL:(NSURL *)originalItemURL
+ withItemAtURL:(NSURL *)newItemURL
+ backupItemName:(NSString *)backupItemName
+ options:(NSFileManagerItemReplacementOptions)options
+ resultingItemURL:(NSURL * _Nullable *)resultingURL
+ error:(NSError * _Nullable *)error;
+
This method should be preferred for Mac OS X 10.6 and newer. Unlike exchangedata(), this function works on APFS.