Merge pull request #228 from microsoft/pete-dev

More daily updates

.github/ISSUE_TEMPLATE/bug_report.md

@@ -4,12 +4,10 @@ about: Create a bug report to help us improve Windows MIDI Services
 title: "[BUG]"
 labels: "bug :lady_beetle:"
 assignees: Psychlist1972
+project: "microsoft/339"
 
 ---
 
-**Applies To**
-If known, is this a bug in a specific tool, in the SDK, API, driver, something else? If unsure, just tell us what you were using when you encountered the bug.
-
 **Describe the bug**
 A clear and concise description of what the bug is.
 
@@ -22,6 +20,9 @@ A clear and concise description of what you expected to happen.
 **Screenshots**
 If applicable, add screenshots to help explain your problem. This is especially helpful for any MIDI tools bug reports, or bug reports related to a third-party product.
 
+
+
+
 **Installer Name or Version**
 If this is an install from a Github release, please provide the version (or the full installer name)
 Example: Windows MIDI Services Developer Preview x64 1.0.24002.1659.exe
@@ -30,8 +31,12 @@ Example: Windows MIDI Services Developer Preview x64 1.0.24002.1659.exe
  - OS: [e.g. Windows 11]
  - OS Build from Settings > System > About: [e.g. 22621.1848]
 
-**Do you wish to contribute a fix for this problem?**
-Yes or No
+**Device information, if this is with an external MIDI device:**
+ - Brand and model and any important details
+ - Driver in use, if you know (Windows MIDI 2.0 class driver, classic MIDI 1.0 class driver, a vendor driver, etc)
+
+**Application Information**
+If this is a third-party application, please include application name and version
 
 **Additional context**
 Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -2,8 +2,9 @@
 name: Feature request
 about: Suggest an idea for this project
 title: "[Feature]"
-labels: feature proposal
 assignees: Psychlist1972
+project: "microsoft/339"
+labels: "feature request :bulb:"
 
 ---
 
@@ -17,6 +18,7 @@ A clear and concise description of what you want to happen.
 A clear and concise description of any alternative solutions or features you've considered.
 
 **Do you wish to contribute a solution?**
+Yes or no
 
 **Additional context**
 Add any other context or screenshots about the feature request here.

build/staging/version/BundleInfo.wxi

@@ -1,4 +1,4 @@
 <Include>
   <?define SetupVersionName="Developer Preview 4" ?>
-  <?define SetupVersionNumber="1.0.24016.1413" ?>
+  <?define SetupVersionNumber="1.0.24017.2041" ?>
 </Include>

get-started/midi-users/docs/midi-console.md

@@ -221,8 +221,20 @@ midi endpoint send-message 0x41234567 0xDEADBEEF --count 15 --pause 2000
 
 In general, we recommend sending messages in hexadecimal format (prefix `0x` followed by 8 hexadecimal digits)as it is easier to visually inspect the information being sent. The 1-4 MIDI words are in order from left to right, from 1 to 4. 
 
+#### Special debug messages
+
+One thing that can be useful is to send otherwise valid UMP messages where the last word is incremented by 1 for each sent message. This helps to validate that all messages were received by your application, and in the correct order. Note that this requires a message type of at least two words. We don't recommend sending Type F stream messages as those have the potential to corrupt data. Instead, a Type 4 MIDI 2.0 channel voice message is usually safer.
+
+```
+midi endpoint send-message 0x41234567 0x00000000 --count 10000 --pause 2 --debug-auto-increment
+```
+
+When sent, you should see messages where the second word is updated from `0x00000000` through `0x00002710` (decimal 10000). We recommend the pause when sending large numbers of messages because a pause of 0 ("send as fast as possible") can flood the buffers with more data than the client may be able to retrieve in time and may result in dropped messages. A warning is displayed when that possibility seems likely.
+
 #### Scheduling messages
 
+> NOTE: In current Developer Preview builds, message scheduling is turned off so the timestamp is ignored. Refer to the release notes.
+
 When sending messages, you have two options for timestamps:
 
 `--offset-microseconds` is used to add a fixed time to each outgoing message so that it is scheduled that far into the future.  

src/api/Abstraction/KSAbstraction/Midi2.KSMidiEndpointManager.cpp

@@ -509,6 +509,13 @@ CMidi2KSMidiEndpointManager::ApplyUserConfiguration(std::wstring deviceInterface
                 endpointProperties.push_back({ {PKEY_MIDI_UserSuppliedEndpointName, DEVPROP_STORE_SYSTEM, nullptr},
                         DEVPROP_TYPE_STRING, static_cast<ULONG>((name.size() + 1) * sizeof(WCHAR)), (PVOID)name.c_str() });
             }
+            else
+            {
+                // delete any existing property value, because it is no longer in the config
+                endpointProperties.push_back({ {PKEY_MIDI_UserSuppliedEndpointName, DEVPROP_STORE_SYSTEM, nullptr},
+                        DEVPROP_TYPE_EMPTY, 0, nullptr });
+            }
+
 
             // Get the user-specified endpoint description
             if (endpointSettings != nullptr && endpointSettings.HasKey(MIDI_CONFIG_JSON_ENDPOINT_USER_SUPPLIED_DESCRIPTION_PROPERTY_KEY))
@@ -518,6 +525,12 @@ CMidi2KSMidiEndpointManager::ApplyUserConfiguration(std::wstring deviceInterface
                 endpointProperties.push_back({ {PKEY_MIDI_UserSuppliedDescription, DEVPROP_STORE_SYSTEM, nullptr},
                         DEVPROP_TYPE_STRING, static_cast<ULONG>((description.size() + 1) * sizeof(WCHAR)), (PVOID)description.c_str() });
             }
+            else
+            {
+                // delete any existing property value, because it is no longer in the config
+                endpointProperties.push_back({ {PKEY_MIDI_UserSuppliedDescription, DEVPROP_STORE_SYSTEM, nullptr},
+                        DEVPROP_TYPE_EMPTY, 0, nullptr });
+            }
 
             // Get the user-specified multiclient override
             if (endpointSettings != nullptr && endpointSettings.HasKey(MIDI_CONFIG_JSON_ENDPOINT_FORCE_SINGLE_CLIENT_PROPERTY_KEY))
@@ -532,6 +545,10 @@ CMidi2KSMidiEndpointManager::ApplyUserConfiguration(std::wstring deviceInterface
                             DEVPROP_TYPE_BOOLEAN, static_cast<ULONG>(sizeof(devPropFalse)), (PVOID)&devPropFalse });
                 }
             }
+            else
+            {
+                // this property was an override, so it should have been set elsewhere earlier
+            }
 
             // apply supported property changes.
             if (endpointProperties.size() > 0)

src/api/Abstraction/MidiSrvAbstraction/Midi2.MidiSrv.cpp

@@ -52,8 +52,9 @@ CMidi2MidiSrv::Initialize(
     // Todo: client side buffering requests to come from some service setting?
     // - See https://github.com/microsoft/MIDI/issues/219 for details
 
-    creationParams.BufferSize = PAGE_SIZE;
+    //creationParams.BufferSize = PAGE_SIZE;  // original
     //creationParams.BufferSize = 512;    // Set this for debugging see https://github.com/microsoft/MIDI/issues/182 for all the drama :)
+    creationParams.BufferSize = PAGE_SIZE * 4;
 
 
     RETURN_IF_FAILED(GetMidiSrvBindingHandle(&bindingHandle));

src/user-tools/midi-console/Midi/AnsiMarkupFormatter.cs

@@ -67,6 +67,11 @@ public static string FormatError(string error)
             return "[red]" + EscapeString(error) + "[/]";
         }
 
+        public static string FormatWarning(string warning)
+        {
+            return "[yellow]" + EscapeString(warning) + "[/]";
+        }
+
         public static string FormatSuccess(string message)
         {
             return "[green]" + EscapeString(message) + "[/]";

src/user-tools/midi-console/Midi/Commands/Endpoint/EndpointCommandSettings.cs

@@ -21,7 +21,7 @@ internal class SendMessageCommandSettings : EndpointCommandSettings
     {
         [LocalizedDescription("ParameterSendMessageDelayBetweenMessages")]
         [CommandOption("-p|--pause|--delay")]
-        [DefaultValue(0)]
+        [DefaultValue(2)]
         public int DelayBetweenMessages { get; set; }
 
         [EnumLocalizedDescription("ParameterSendMessageWordFormat", typeof(MidiWordDataFormat))]

src/user-tools/midi-console/Midi/Commands/Endpoint/EndpointMonitorCommand.cs

@@ -27,6 +27,8 @@ public struct ReceivedMidiMessage
         public UInt32 Word1;
         public UInt32 Word2;
         public UInt32 Word3;
+
+        public bool HasError;
     }
 
 
@@ -80,6 +82,11 @@ public sealed class Settings : MessageListenerCommandSettings
             [DefaultValue(CaptureFieldDelimiter.Space)]
             public CaptureFieldDelimiter FieldDelimiter { get; set; }
 
+            [EnumLocalizedDescription("ParameterCaptureMessagesFieldDebugWarnSkippedIncrement", typeof(CaptureFieldDelimiter))]
+            [CommandOption("-w|--debug-warn-skipped-increment")]
+            [DefaultValue(false)]
+            public bool WarnIfSkippedIncrementMessage { get; set; }
+
             //[EnumLocalizedDescription("ParameterMonitorEndpointSkipToKeepUp", typeof(CaptureFieldDelimiter))]
             //[CommandOption("-k|--keep-up-display")]
             //[DefaultValue(false)]
@@ -160,7 +167,7 @@ public override int Execute(CommandContext context, Settings settings)
 
             MidiMessageTable displayTable = new MidiMessageTable(settings.Verbose);
 
-            bool capturingToFile = false;
+        //    bool capturingToFile = false;
 
             string endpointId = string.Empty;
 
@@ -210,15 +217,13 @@ public override int Execute(CommandContext context, Settings settings)
 
                     AnsiConsole.MarkupLine("Capturing to " + AnsiMarkupFormatter.FormatFileName(fileName));
 
-                    capturingToFile = true;
+                    //capturingToFile = true;
                 }
                 else
                 {
                     //captureWriter = null;
                 }
 
-                AnsiConsole.MarkupLine(Strings.MonitorPressEscapeToStopMonitoringMessage);
-                AnsiConsole.WriteLine();
 
                 UInt64 startTimestamp = 0;
                 UInt64 lastReceivedTimestamp = 0;
@@ -232,8 +237,10 @@ public override int Execute(CommandContext context, Settings settings)
                 MonitorEndpointConnectionStatusInTheBackground(endpointId);
 
                 bool continueWaiting = true;
-                UInt64 outOfOrderMessageCount = 0;
+
+                UInt64 outOfTimestampOrderMessageCount = 0;
 
+                uint numberOfSkippedDebugMessages = 0;
 
                 // open the connection
                 if (connection.Open())
@@ -244,6 +251,9 @@ public override int Execute(CommandContext context, Settings settings)
                     {
                         UInt64 lastMessageTimestamp = 0;
 
+                        UInt32 lastReceivedDebugWord = 0;
+                        UInt32 currentDebugWord = 0;
+
                         connection.MessageReceived += (s, e) =>
                         {
                             // this captures stats to tell us how long it takes to receive and queue messages
@@ -257,7 +267,6 @@ public override int Execute(CommandContext context, Settings settings)
                             // helps prevent any race conditions with main loop and its output
                             if (!continueWaiting) return;
 
-                            //Console.WriteLine("DEBUG: MessageReceived");
                             index++;
 
                             var receivedMessage = new ReceivedMidiMessage()
@@ -269,13 +278,45 @@ public override int Execute(CommandContext context, Settings settings)
 
                             if (e.Timestamp < lastMessageTimestamp)
                             {
-                                outOfOrderMessageCount++;
+                                outOfTimestampOrderMessageCount++;
+                                receivedMessage.HasError = true;
                             }
 
                             lastMessageTimestamp = e.Timestamp;
 
 
-                            receivedMessage.NumWords = e.FillWords(out receivedMessage.Word0, out receivedMessage.Word1, out receivedMessage.Word2, out receivedMessage.Word3);
+                            receivedMessage.NumWords = e.FillWords(
+                                out receivedMessage.Word0, 
+                                out receivedMessage.Word1, 
+                                out receivedMessage.Word2, 
+                                out receivedMessage.Word3);
+
+
+                            switch (receivedMessage.NumWords)
+                            {
+                                case 1: break; // 1 word messages don't auto-increment
+                                case 2:
+                                    currentDebugWord = receivedMessage.Word1;
+                                    break;
+                                case 3:
+                                    currentDebugWord = receivedMessage.Word2;
+                                    break;
+                                case 4:
+                                    currentDebugWord = receivedMessage.Word3;
+                                    break;
+                            }
+
+                            if (settings.WarnIfSkippedIncrementMessage && index > 1)
+                            {
+                                if (currentDebugWord != lastReceivedDebugWord + 1)
+                                {
+                                    numberOfSkippedDebugMessages += (currentDebugWord - lastReceivedDebugWord - 1);
+                                    receivedMessage.HasError = true;
+                                }
+
+                                lastReceivedDebugWord = currentDebugWord;
+                            }
+
 
                             lock (m_receivedMessagesQueue)
                             {
@@ -295,7 +336,7 @@ public override int Execute(CommandContext context, Settings settings)
                     messageListener.Start();
 
 
-                    const UInt32 minMessagesToLagBeforeSkip = 10;
+                    //const UInt32 minMessagesToLagBeforeSkip = 10;
 
                     // Console display background thread -----------------------------------------------------
                     var messageConsoleDisplay = new Thread(() =>
@@ -321,8 +362,7 @@ public override int Execute(CommandContext context, Settings settings)
                                 }
 
                                 if (lastReceivedTimestamp == 0)
-                                {
-                                    // gets timestamp of first message we receive and uses that so all others are an offset from previous message
+                                {                                   
                                     lastReceivedTimestamp = message.ReceivedTimestamp;
                                 }
 
@@ -333,7 +373,7 @@ public override int Execute(CommandContext context, Settings settings)
 
                                 // calculate offset from the last message received
                                 var offsetMicroseconds = MidiClock.ConvertTimestampToMicroseconds(
-                                message.ReceivedTimestamp - lastReceivedTimestamp);
+                                    message.ReceivedTimestamp - lastReceivedTimestamp);
 
                                 // set our last received so we can calculate offsets
                                 lastReceivedTimestamp = message.ReceivedTimestamp;
@@ -441,6 +481,8 @@ public override int Execute(CommandContext context, Settings settings)
 
 
                     // Main UI loop and moving messages to output queues -----------------------------------------
+                    AnsiConsole.MarkupLine(Strings.MonitorPressEscapeToStopMonitoringMessage);
+                    AnsiConsole.WriteLine();
                     while (continueWaiting)
                     {
                         if (Console.KeyAvailable)
@@ -513,7 +555,7 @@ public override int Execute(CommandContext context, Settings settings)
                     // calculate total receive time, not total display time
 
                     UInt64 totalTicks = lastMessageReceivedEventTimestamp - firstMessageReceivedEventTimestamp;
-                    double totalSeconds = MidiClock.ConvertTimestampToMilliseconds(totalTicks) / 1000;
+                    double totalSeconds = MidiClock.ConvertTimestampToMilliseconds(totalTicks) / 1000.0;
 
                     message += $" and processed over [steelblue1]{totalSeconds.ToString("N2")}[/] seconds, ";
 
@@ -524,7 +566,7 @@ public override int Execute(CommandContext context, Settings settings)
                     if (averageMicroseconds > 1000000)
                     {
                         // display in seconds
-                        double averageSeconds = averageMicroseconds / 1000000;
+                        double averageSeconds = averageMicroseconds / 1000000.0;
                         message += $"[steelblue1]{averageSeconds.ToString("N4")} s[/] per message.";
                     }
                     else if (averageMilliseconds > 1)
@@ -541,11 +583,11 @@ public override int Execute(CommandContext context, Settings settings)
                     AnsiConsole.MarkupLine(message);
                 }
 
-                if (outOfOrderMessageCount > 0)
+                if (outOfTimestampOrderMessageCount > 0)
                 {
-                    string message = "❎ " + outOfOrderMessageCount.ToString();
+                    string message = "❎ " + outOfTimestampOrderMessageCount.ToString();
 
-                    if (outOfOrderMessageCount > 1)
+                    if (outOfTimestampOrderMessageCount > 1)
                     {
                         // multiple messages out of order
                         message += " messages received out of sent timestamp order.";
@@ -563,7 +605,34 @@ public override int Execute(CommandContext context, Settings settings)
                     AnsiConsole.MarkupLine("✅ No messages received out of expected timestamp order.");
                 }
 
-          //      AnsiConsole.MarkupLine($"{MidiClock.ConvertTimestampToMilliseconds(displayTable.TotalRenderingTicks).ToString("N4")} milliseconds spent just on output rendering.");
+                if (settings.WarnIfSkippedIncrementMessage)
+                {
+                    if (numberOfSkippedDebugMessages > 0)
+                    {
+                        string message = "❎ " + numberOfSkippedDebugMessages.ToString();
+
+                        if (numberOfSkippedDebugMessages > 1)
+                        {
+                            // multiple messages out of order
+                            message += " in-line messages skipped (assuming they were sent with debug words).";
+                        }
+                        else
+                        {
+                            // single message out of order
+                            message += " in-line message skipped (assuming it was sent with debug words).";
+                        }
+
+                        AnsiConsole.MarkupLine(AnsiMarkupFormatter.FormatError(message));
+                    }
+                    else
+                    {
+                        AnsiConsole.MarkupLine("✅ No messages skipped.");
+                    }
+                }
+
+
+
+                //      AnsiConsole.MarkupLine($"{MidiClock.ConvertTimestampToMilliseconds(displayTable.TotalRenderingTicks).ToString("N4")} milliseconds spent just on output rendering.");
 
                 if (captureWriter != null)
                 {