fclose: Check for malloc failure when closing temp files.

Previously, the code for closing a temporary file assumed that malloc would succeed. If it did not, the code would trash memory and (at least in my testing) crash the system. Now it checks for and handles malloc failures, although they will still lead to the temporary file not being deleted.

Here is a test program illustrating the problem:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
        FILE *f = tmpfile();
        if (!f)
                return 0;

        void *p;
        do {
                p = malloc(8*1024);
        } while (p);

        fclose(f);
}
This commit is contained in:
Stephen Heumann 2024-02-20 22:20:17 -06:00
parent 16c7952648
commit 7384c82667
1 changed files with 10 additions and 4 deletions

View File

@ -129,12 +129,18 @@ cl3a ldy #FILE_flag if the file was opened by tmpfile then
and #_IOTEMPFILE
beq cl3d
ph4 #nameBuffSize p = malloc(nameBuffSize)
jsl malloc grPathname = p
sta p dsPathname = p+2
jsl malloc
sta p
stx p+2
sta grPathname
ora p+2 if p == NULL then
bne cl3aa
lda #EOF flag error
sta err
bra cl3d just close the file
cl3aa lda p
sta grPathname grPathname = p
stx grPathname+2
clc
clc dsPathname = p+2
adc #2
bcc cl3b
inx