Disregard operand address, part 1

SourceGen puts a lot of effort into connecting address operands to
internal offsets and external symbols.  Sometimes this is undesirable.
For example, a BIT or conditional branch instruction is sometimes used
as a dummy when there are multiple entry points that vary only in a
status flag or register value.  The address in the BIT or branch is
either irrelevant or never actually referenced, so auto-generating a
label for it is annoying.

Having a way to tell the disassembler to disregard an address operand,
so that it won't generate an auto-label or be included in the list of
cross-references, provides a simple solution to this.

The operand's file offset is determined during the code analysis
pass.  Setting it to -1 makes the operand look like an external address.
If we disable the project/platform symbol lookup for that instruction
as well, we can avoid label generation and cross-references.

We also need to prevent the 65816 data bank fixup code from restoring
it, and the relocation data code from doing its thing.

The flag is implemented as a new "misc flags" table, which holds a
set of bit flags for every file offset.  The "disregard operand address"
flag is set on the opcode byte of instructions.  The flags are expected
to be used infrequently, so they're currently output as a list of
(offset, integer) pairs.

The flag is set in the Edit Instruction Operand dialog, which has a
new checkbox.  The checkbox is disabled for immediate operands.

(issue #174)

SourceGen/AsmGen/AsmTass64.cs

@@ -397,6 +397,7 @@ namespace SourceGen.AsmGen {
                     offset += len;
                 } else {
                     // data items
+                    Debug.Assert(attr.Length > 0);
                     offset += attr.Length;
                 }
             }

SourceGen/CodeAnalysis.cs

@@ -159,6 +159,11 @@ namespace SourceGen {
         /// </summary>
         private StatusFlags[] mStatusFlagOverrides;
 
+        /// <summary>
+        /// Miscellaneous flags, one entry per byte.
+        /// </summary>
+        private DisasmProject.MiscFlag[] mMiscFlags;
+
         /// <summary>
         /// Initial status flags to use at entry points.
         /// </summary>
@@ -178,6 +183,10 @@ namespace SourceGen {
         /// <summary>
         /// Constructor.
         /// </summary>
+        /// <remarks>
+        /// This takes individual arrays as arguments, rather than the project file, to
+        /// ensure that we're restricting our analysis to a specific set of data tables.
+        /// </remarks>
         /// <param name="data">65xx code stream.</param>
         /// <param name="cpuDef">CPU definition to use when interpreting code.</param>
         /// <param name="anattribs">Anattrib array.  Expected to be newly allocated, all
@@ -186,20 +195,23 @@ namespace SourceGen {
         /// <param name="atags">Analyzer tags, one per byte.</param>
         /// <param name="statusFlagOverrides">Status flag overrides for instruction-start
         ///    bytes.</param>
+        /// <param name="miscFlags">Miscellaneous flags, one per byte.</param>
         /// <param name="entryFlags">Status flags to use at code entry points.</param>
         /// <param name="scriptMan">Extension script manager.</param>
         /// <param name="parms">Analysis parameters.</param>
         /// <param name="debugLog">Object that receives debug log messages.</param>
         public CodeAnalysis(byte[] data, CpuDef cpuDef, Anattrib[] anattribs,
                 AddressMap addrMap, AnalyzerTag[] atags, StatusFlags[] statusFlagOverrides,
-                StatusFlags entryFlags, ProjectProperties.AnalysisParameters parms,
+                DisasmProject.MiscFlag[] miscFlags, StatusFlags entryFlags,
-                ScriptManager scriptMan, DebugLog debugLog) {
+                ProjectProperties.AnalysisParameters parms,ScriptManager scriptMan,
+                DebugLog debugLog) {
             mFileData = data;
             mCpuDef = cpuDef;
             mAnattribs = anattribs;
             mAddrMap = addrMap;
             mAnalyzerTags = atags;
             mStatusFlagOverrides = statusFlagOverrides;
+            mMiscFlags = miscFlags;
             mEntryFlags = entryFlags;
             mScriptManager = scriptMan;
             mAnalysisParameters = parms;
@@ -1015,7 +1027,15 @@ namespace SourceGen {
                 int operandOffset = mAddrMap.AddressToOffset(offset,
                     mAnattribs[offset].OperandAddress);
                 if (operandOffset >= 0) {
-                    mAnattribs[offset].OperandOffset = operandOffset;
+                    bool disregard =
+                        (mMiscFlags[offset] & DisasmProject.MiscFlag.DisregardOperandAddress) != 0;
+                    if (!disregard) {
+                        mAnattribs[offset].OperandOffset = operandOffset;
+                    } else {
+                        //Debug.WriteLine("Disregarding operand address at +" +
+                        //    offset.ToString("x6"));
+                        mAnattribs[offset].OperandOffset = -1;
+                    }
 
                     // Set a flag if this is a direct offset.  This is used when tracing
                     // through jump instructions, as we can't necessarily decode an indirect
@@ -1399,7 +1419,8 @@ namespace SourceGen {
         /// </summary>
         /// <remarks>
         /// This is of questionable value when we have reliable relocation data.  OTOH it's
-        /// pretty quick even on very large files.
+        /// pretty quick even on very large files.  This is executed as a follow-up pass after
+        /// code analysis completes.
         /// </remarks>
         public void ApplyDataBankRegister(Dictionary<int, DbrValue> userValues,
                 Dictionary<int, DbrValue> dbrChanges) {
@@ -1429,6 +1450,9 @@ namespace SourceGen {
                 if (mAnattribs[offset].IsNonAddressable) {
                     continue;
                 }
+                if ((mMiscFlags[offset] & DisasmProject.MiscFlag.DisregardOperandAddress) != 0) {
+                    continue;
+                }
                 if (curVal == DbrValue.UNKNOWN) {
                     // On first encounter with addressable memory, init curVal so B=K.
                     curVal = (byte)(mAddrMap.OffsetToAddress(offset) >> 16);

SourceGen/DataAnalysis.cs

@@ -193,8 +193,12 @@ namespace SourceGen {
                                 //Debug.WriteLine("REL +" + offset.ToString("x6") + " " +
                                 //    reloc.Value.ToString("x6") + " vs. " +
                                 //    attr.OperandAddress.ToString("x6"));
-                                WeakSymbolRef.Part part = ShiftToPart(reloc.Shift);
+                                bool disregard = (mProject.MiscFlags[offset] &
-                                SetDataTarget(offset, attr.Length, relOperandOffset, part);
+                                    DisasmProject.MiscFlag.DisregardOperandAddress) != 0;
+                                if (!disregard) {
+                                    WeakSymbolRef.Part part = ShiftToPart(reloc.Shift);
+                                    SetDataTarget(offset, attr.Length, relOperandOffset, part);
+                                }
                                 continue;
                             }
                         }

SourceGen/DisasmProject.cs

@@ -33,15 +33,14 @@ namespace SourceGen {
         // Arbitrary 1MB limit.  Could be increased to 16MB if performance is acceptable.
         public const int MAX_DATA_FILE_SIZE = 1 << 20;
 
-        // File magic.
+        public static readonly long FILE_MAGIC = 6982516645493599905;
-        private const long MAGIC = 6982516645493599905;
 
 
         #region Data that is saved to the project file
         // All data held by structures in this section are persistent, and will be
         // written to the project file.  Anything not in this section may be discarded
         // at any time.  Smaller items are kept in arrays, with one entry per byte
-        // of file data.
+        // of file data, for fast access during code analysis.
 
         /// <summary>
         /// Length of input data.  (This is redundant with FileData.Length while in memory,
@@ -69,6 +68,20 @@ namespace SourceGen {
         /// </summary>
         public StatusFlags[] StatusFlagOverrides { get; private set; }
 
+        /// <summary>
+        /// Miscellaneous flags.
+        /// </summary>
+        /// <remarks>
+        /// These are expected to be set rarely.  They are serialized integers,
+        /// so the flag enumeration can be renamed, but DO NOT re-use flag values.
+        /// </remarks>
+        public MiscFlag[] MiscFlags { get; private set; }
+        [Flags]
+        public enum MiscFlag : byte {
+            None = 0,
+            DisregardOperandAddress = 1 << 0,
+        }
+
         /// <summary>
         /// End-of-line comments.  Empty string means "no comment".
         /// </summary>
@@ -279,6 +292,9 @@ namespace SourceGen {
             // Default value is "unspecified" for all bits.
             StatusFlagOverrides = new StatusFlags[fileDataLen];
 
+            // Default value is 0 (all flags clear).
+            MiscFlags = new MiscFlag[fileDataLen];
+
             Comments = new string[fileDataLen];
 
             // Populate with empty strings so we don't have to worry about null refs.
@@ -853,7 +869,7 @@ namespace SourceGen {
                 reanalysisTimer.StartTask("CodeAnalysis.Analyze");
 
                 CodeAnalysis ca = new CodeAnalysis(mFileData, CpuDef, mAnattribs, AddrMap,
-                    AnalyzerTags, StatusFlagOverrides, ProjectProps.EntryFlags,
+                    AnalyzerTags, StatusFlagOverrides, MiscFlags, ProjectProps.EntryFlags,
                     ProjectProps.AnalysisParams, mScriptManager, debugLog);
 
                 ca.Analyze();
@@ -1451,10 +1467,16 @@ namespace SourceGen {
                     // symbol with the same name from being visible.  Using the full symbol
                     // table is potentially a tad less efficient than looking for a match
                     // exclusively in project/platform symbols, but it's the correct thing to do.
-                    OpDef op = CpuDef.GetOpDef(FileData[offset]);
+                    bool disregard = (MiscFlags[offset] & MiscFlag.DisregardOperandAddress) != 0;
-                    accType = op.MemEffect;
+                    if (disregard) {
-                    address = attr.OperandAddress;
+                        address = -1;
-                    sym = SymbolTable.FindNonVariableByAddress(address, accType);
+                        sym = null;
+                    } else {
+                        OpDef op = CpuDef.GetOpDef(FileData[offset]);
+                        accType = op.MemEffect;
+                        address = attr.OperandAddress;
+                        sym = SymbolTable.FindNonVariableByAddress(address, accType);
+                    }
                 } else if ((attr.IsDataStart || attr.IsInlineDataStart) &&
                         attr.DataDescriptor != null && attr.DataDescriptor.IsNumeric &&
                         attr.DataDescriptor.FormatSubType == FormatDescriptor.SubType.Address) {
@@ -2502,6 +2524,17 @@ namespace SourceGen {
                             AddAffectedLine(affectedOffsets, offset);
                         }
                         break;
+                    case UndoableChange.ChangeType.SetMiscFlags: {
+                            MiscFlag curFlags = MiscFlags[offset];
+                            if ((MiscFlag)curFlags != (MiscFlag)oldValue) {
+                                Debug.WriteLine("GLITCH: old misc flags value mismatch ($" +
+                                    curFlags.ToString("x") + " vs " +
+                                    ((MiscFlag)oldValue).ToString("x") + ")");
+                            }
+                            MiscFlags[offset] = (MiscFlag)newValue;
+                            // Scope of effects could be large.
+                        }
+                        break;
                     case UndoableChange.ChangeType.SetProjectProperties: {
                             bool needPlatformSymReload = !CommonUtil.Container.StringListEquals(
                                 ((ProjectProperties)oldValue).PlatformSymbolFileIdentifiers,
@@ -2577,6 +2610,7 @@ namespace SourceGen {
                         }
                         break;
                     default:
+                        Debug.Assert(false, "unhandled case " + uc.Type);
                         break;
                 }
                 if (needReanalysis < uc.ReanalysisRequired) {

SourceGen/MainController.cs

@@ -2943,6 +2943,23 @@ namespace SourceGen {
                 Debug.WriteLine("No change to project symbol");
             }
 
+            if (mProject.MiscFlags[offset] != 0 || dlg.DisregardOperandAddress) {
+                // Flags are defined, or the "disregard" checkbox is checked.  See if
+                // something has changed.
+                DisasmProject.MiscFlag curFlags = mProject.MiscFlags[offset];
+                bool curVal = (curFlags & DisasmProject.MiscFlag.DisregardOperandAddress) != 0;
+                bool newVal = dlg.DisregardOperandAddress;
+                if (curVal != newVal) {
+                    DisasmProject.MiscFlag newFlags =
+                        (curFlags & ~DisasmProject.MiscFlag.DisregardOperandAddress) |
+                        (newVal ? DisasmProject.MiscFlag.DisregardOperandAddress : 0);
+                    Debug.Assert(curFlags != newFlags);
+                    UndoableChange uc = UndoableChange.CreateMiscFlagsChange(offset, curFlags,
+                        newFlags);
+                    cs.Add(uc);
+                }
+            }
+
             Debug.WriteLine("EditInstructionOperand: " + cs.Count + " changes");
             if (cs.Count != 0) {
                 ApplyUndoableChanges(cs);
@@ -5234,6 +5251,13 @@ namespace SourceGen {
             if (attr.NoContinueScript) {
                 sb.AppendLine("\"No-continue\" flag set by script");
             }
+            if (attr.DoesNotBranch) {
+                sb.AppendLine("Conditional branch is never taken");
+            }
+            if ((mProject.MiscFlags[line.FileOffset] &
+                    DisasmProject.MiscFlag.DisregardOperandAddress) != 0) {
+                sb.AppendLine("Operand address is being disregarded");
+            }
             if (attr.HasAnalyzerTag) {
                 sb.Append("\u2022 Analyzer Tags: ");
                 for (int i = 0; i < line.OffsetSpan; i++) {

SourceGen/ProjectFile.cs

@@ -56,7 +56,7 @@ namespace SourceGen {
         // ignore stuff that's in one side but not the other.  However, if we're opening a
         // newer file in an older program, it's worth letting the user know that some stuff
         // may get lost as soon as they save the file.
-        public const int CONTENT_VERSION = 6;
+        public const int CONTENT_VERSION = 7;
 
         // Max JSON file length.
         internal const int MAX_JSON_LENGTH = 64 * 1024 * 1024;
@@ -470,6 +470,7 @@ namespace SourceGen {
         public List<SerAddressMapEntry> AddressMap { get; set; }
         public List<SerTypeHintRange> TypeHints { get; set; }
         public Dictionary<string, int> StatusFlagOverrides { get; set; }
+        public Dictionary<string, int> MiscFlags { get; set; }
         public Dictionary<string, string> Comments { get; set; }
         public Dictionary<string, SerMultiLineComment> LongComments { get; set; }
         public Dictionary<string, SerMultiLineComment> Notes { get; set; }
@@ -533,6 +534,16 @@ namespace SourceGen {
                 spf.StatusFlagOverrides.Add(i.ToString(), proj.StatusFlagOverrides[i].AsInt);
             }
 
+            // Write flags as an integer.  A string of enumerations would have some advantages,
+            // but simplicity is not among them.
+            spf.MiscFlags = new Dictionary<string, int>();
+            for (int i = 0; i < proj.MiscFlags.Length; i++) {
+                if (proj.MiscFlags[i] == DisasmProject.MiscFlag.None) {
+                    continue;
+                }
+                spf.MiscFlags.Add(i.ToString(), (int)proj.MiscFlags[i]);
+            }
+
             // Convert Comments to serializable form.
             spf.Comments = new Dictionary<string, string>();
             for (int i = 0; i < proj.Comments.Length; i++) {
@@ -761,7 +772,7 @@ namespace SourceGen {
             }
 
             // Deserialize status flag overrides.
-            foreach (KeyValuePair<string,int> kvp in spf.StatusFlagOverrides) {
+            foreach (KeyValuePair<string, int> kvp in spf.StatusFlagOverrides) {
                 if (!ParseValidateKey(kvp.Key, spf.FileDataLength,
                         Res.Strings.PROJECT_FIELD_STATUS_FLAGS, report, out int intKey)) {
                     continue;
@@ -961,6 +972,16 @@ namespace SourceGen {
                 }
             }
 
+            // Deserialize misc flags.  These were added in 1.10.
+            if (spf.MiscFlags != null) {
+                foreach (KeyValuePair<string, int> kvp in spf.MiscFlags) {
+                    if (!ParseValidateKey(kvp.Key, spf.FileDataLength,
+                            Res.Strings.PROJECT_FIELD_MISC_FLAGS, report, out int intKey)) {
+                        continue;
+                    }
+                    proj.MiscFlags[intKey] = (DisasmProject.MiscFlag)kvp.Value;
+                }
+            }
 
             return true;
         }

SourceGen/Res/Strings.xaml

@@ -170,6 +170,7 @@ limitations under the License.
     <system:String x:Key="str_ProjectFieldDbrValue">DBR value</system:String>
     <system:String x:Key="str_ProjectFieldLongComment">long comment</system:String>
     <system:String x:Key="str_ProjectFieldLvTable">local variable table</system:String>
+    <system:String x:Key="str_ProjectFieldMiscFlags">misc flag</system:String>
     <system:String x:Key="str_ProjectFieldNote">note</system:String>
     <system:String x:Key="str_ProjectFieldOperandFormat">operand format</system:String>
     <system:String x:Key="str_ProjectFieldRelocData">reloc data</system:String>

SourceGen/Res/Strings.xaml.cs

@@ -321,6 +321,8 @@ namespace SourceGen.Res {
             (string)Application.Current.FindResource("str_ProjectFieldLongComment");
         public static string PROJECT_FIELD_LV_TABLE =
             (string)Application.Current.FindResource("str_ProjectFieldLvTable");
+        public static string PROJECT_FIELD_MISC_FLAGS =
+            (string)Application.Current.FindResource("str_ProjectFieldMiscFlags");
         public static string PROJECT_FIELD_NOTE =
             (string)Application.Current.FindResource("str_ProjectFieldNote");
         public static string PROJECT_FIELD_OPERAND_FORMAT =

SourceGen/UndoableChange.cs

@@ -104,6 +104,9 @@ namespace SourceGen {
             // Changes the note.
             SetNote,
 
+            // Sets one or more flags.
+            SetMiscFlags,
+
             // Updates project properties.
             SetProjectProperties,
 
@@ -498,6 +501,31 @@ namespace SourceGen {
             return uc;
         }
 
+        /// <summary>
+        /// Creates an UndoableChange for a note update.
+        /// </summary>
+        /// <param name="offset">Affected offset.</param>
+        /// <param name="oldFlags">Current flags.</param>
+        /// <param name="newFlags">New flags.</param>
+        /// <returns>Change record.</returns>
+        public static UndoableChange CreateMiscFlagsChange(int offset,
+                DisasmProject.MiscFlag oldFlags, DisasmProject.MiscFlag newFlags) {
+            if (oldFlags == newFlags) {
+                Debug.WriteLine("No-op flags change at +" + offset.ToString("x6") +
+                    ": " + oldFlags);
+            }
+
+            UndoableChange uc = new UndoableChange();
+            uc.Type = ChangeType.SetMiscFlags;
+            uc.Offset = offset;
+            uc.OldValue = oldFlags;
+            uc.NewValue = newFlags;
+            // Actual requirement could vary based on flag.  These should be rare, so for now
+            // just do the full analysis.
+            uc.ReanalysisRequired = ReanalysisScope.CodeAndData;
+            return uc;
+        }
+
         /// <summary>
         /// Creates an UndoableChange for a change to the project properties.
         /// </summary>

SourceGen/WpfGui/EditInstructionOperand.xaml

@@ -103,6 +103,7 @@ limitations under the License.
                         <RowDefinition Height="Auto"/>
                         <RowDefinition Height="Auto"/>
                         <RowDefinition Height="Auto"/>
+                        <RowDefinition Height="*"/>
                         <RowDefinition Height="Auto"/>
                     </Grid.RowDefinitions>
                     <RadioButton Name="formatSymbolButton" Grid.Column="0" Grid.Row="0" GroupName="Main"
@@ -142,6 +143,10 @@ limitations under the License.
                         <TextBlock Grid.Column="1" Grid.Row="1"
                                    Text="{Binding SymbolValueDecimal, FallbackValue=16777215}"/>
                     </Grid>
+
+                    <CheckBox Grid.Column="0" Grid.Row="4" Content="Disregard operand address"
+                              IsChecked="{Binding DisregardOperandAddress}"
+                              IsEnabled="{Binding IsDisregardEnabled}"/>
                 </Grid>
             </Grid>
         </GroupBox>

SourceGen/WpfGui/EditInstructionOperand.xaml.cs

@@ -62,6 +62,16 @@ namespace SourceGen.WpfGui {
         /// </summary>
         public DefSymbol ProjectSymbolResult { get; private set; }
 
+        /// <summary>
+        /// If true, the operand should not be treated as an address during analysis.
+        /// </summary>
+        public bool DisregardOperandAddress {
+            get { return mDisregardOperandAddress; }
+            set { mDisregardOperandAddress = value; OnPropertyChanged(); }
+        }
+        private bool mDisregardOperandAddress;
+
+        public bool IsDisregardEnabled { get; private set; }
 
         /// <summary>
         /// Updated label.
@@ -184,6 +194,13 @@ namespace SourceGen.WpfGui {
                 // For BlockMove this will have both parts.
                 mOperandValue = mOpDef.GetOperand(project.FileData, offset, attr.StatusFlags);
             }
+
+            // Disable the "disregard" checkbox for immediate operands, unless it's already set
+            // (so they can disable it if it got set some weird way).
+            IsDisregardEnabled = DisregardOperandAddress || !(mOpDef.IsImmediate ||
+                mOpDef.AddrMode == OpDef.AddressMode.BlockMove);
+            DisregardOperandAddress =
+                (mProject.MiscFlags[offset] & DisasmProject.MiscFlag.DisregardOperandAddress) != 0;
         }
 
         private void Window_Loaded(object sender, RoutedEventArgs e) {

docs/sgmanual/analysis.html

@@ -49,12 +49,13 @@ method in <code>DisasmProject.cs</code>):</p>
     doing a partial re-analysis, this step will just clone a copy of the
     Anattrib array that was made at this point in a previous run.  (The
     code analysis pass is described in more detail below.)</li>
+  <li>For 65816 code, factor in data bank register changes.</li>
   <li>Apply user-specified labels to Anattribs.</li>
   <li>Apply user-specified format descriptors.  These are the instruction
     and data operand formats.</li>
   <li>Run the data analyzer.  This looks for patterns in uncategorized
     data, and connects instruction and data operands to target offsets.
-    The "nearby label" stuff is handled here.  Auto-labels are generated
+    The "nearby targets" stuff is handled here.  Auto-labels are generated
     for references to internal addresses.  All of the results are
     stored in the Anattribs array.  (The data analysis pass is described in
     more detail below.)</li>
@@ -76,13 +77,14 @@ method in <code>DisasmProject.cs</code>):</p>
     referenced.</li>
   <li>If annotated auto-labels are enabled, the simple labels are
     replaced with the annotated versions here.  (This can't be done earlier
-    because the annotations are generated from the cross-reference data.)</li>
+    because the fancier annotations are generated from the cross-reference
+    usage data.)</li>
   <li>In a debug build, some validity checks are performed.</li>
 </ul>
 
 <p>Once analysis is complete, a line-by-line display list is generated
 by walking through the annotated file data.  Most of the actual text
-isn't rendered until they're needed.  For complicated multi-line items
+isn't rendered until it's first needed.  For complicated multi-line items
 like string operands, the formatted text must be generated to know how
 many lines it will occupy, so it's done immediately and cached for re-use
 on subsequent runs.</p>
@@ -280,15 +282,18 @@ and then the target locations tagged as code entry points.</p>
 operands with the Data Bank Register ("B") to form a 24-bit address.
 SourceGen can't automatically determine what the register holds, so it
 assumes that it's equal to the program bank register ("K"), and provides
-a way to override the value.</p>
+a way to override the value.  With "Smart PLB Handling" enabled, common
+patterns like <code>LDA #$E0 / PHA / PLB</code> are recognized
+automatically.</p>
 
 
 <h3 id="extension-scripts">Extension Scripts</h3>
 
 <p>Extension scripts can mark data that follows a JSR, JSL, or BRK as inline
 data, or change the format of nearby data or instructions.  The first
-time a JSR/JSL/BRK instruction is encountered, all loaded extension scripts
+time a JSR/JSL/BRK instruction is encountered in the current analysis pass,
-that implement the appropriate interface are offered a chance to act.</p>
+all loaded extension scripts that implement the appropriate interface are
+offered a chance to act.</p>
 
 <p>The first script that applies a format wins.  Attempts to re-format
 instructions or data that have already been formatted will fail.  This rule
@@ -305,8 +310,8 @@ before code analysis starts.)</p>
 
 
 <h2 id="data-analysis">Data Analysis</h2>
-<p>The data analyzer performs two tasks.  It matches operands with
+<p>The data analyzer performs two main tasks.  It links operands to
-offsets, and it analyzes uncategorized data.  This behavior can be
+file offsets, and it analyzes uncategorized data.  This behavior can be
 modified in the
 <a href="settings.html#project-properties">project properties</a>.</p>