Visual Studio For Mac Debug Unit Test

VISUAL STUDIO FOR MAC Use MacinCloud to develop feature-rich applications with Visual Studio for Mac. Use modern debugging features to debug cross-platform apps. For remote building COLLABORATE. Collaborate with others using Git repos, built directly in to Visual Studio TEST. Write unit tests, use UI test generation features, and test. Visual Studio 2017 for Mac Release Notes. We fixed an issue where adding a unit test, Visual Studio for Mac will crash with a stack trace that seems to try to recursively add the unit test. Visual Studio for Mac debugger often crashes when debugging Xamarin.iOS. Live Unit Testing is present in the Enterprise edition of Visual Studio 2017 and it’s available for C# and VB projects that target the.NET Framework. [UPDATE 5/23/2017: We now also support.NET Core Framework. Check out this post announcing.NET Core 2.0 Preview 1 to learn about it.] This is a.

Live Unit Testing is present in the Enterprise edition of Visual Studio 2017 and it’s available for C# and VB projects that target the .NET Framework. [UPDATE 5/23/2017 : We now also support .NET Core Framework. Check out this post announcing .NET Core 2.0 Preview 1 to learn about it.] This is a more comprehensive blog than the one we published in November. In this blog, we focus on all capabilities including some that were not mentioned in the earlier blog. It also includes an updated video that demos all these capabilities. FAQsare covered as well at the end.

This is a productivity feature, which provides real-time feedback directly in the editor on how code changes are impacting your unit tests and your code coverage. This will help you maintain quality by keeping the tests passing as you make changes. It will also remind you when you need to write additional unit tests as you are making bug fixes and adding features.

The Live Unit Testing capabilities in Visual Studio 2017 Enterprise are highlighted below:

Easy to Start, Stop, Pause or Restart

To enable Live Unit Testing, go to the Test command of the top-level menu bar, choose “Live Unit Testing”, then “Start”:

At any time, you can temporarily pause or completely stop Live Unit Testing; for example, when you are in the middle of a refactoring, and you know that your tests will be broken for a while. It is as simple as opening the Test menu, selecting Live Unit Testing, and choosing one of the following options:

Visual studio debugger download
  • Pause to suspend Live Unit Testing. When Live Unit Testing is paused, you do not see any coverage visualization in the editor, but all the data that is collected so far is preserved. When you are ready to resume, select Continue from the Live Unit Testing menu. Live Unit Testing will do the necessary work to catch up with all the edits that have been made while it was paused and will update the glyphs appropriately.
  • Stop to completely stop Live Unit Testing. When Live Unit Testing is started after it had been stopped, it takes longer to show the glyphs than when it was paused and resumed. This is because it loses all data when it is stopped.
  • Restart is the equivalent to selecting Stop and immediately selecting Start to start Live Unit Testing again.

View coverage information in the editor as you type

Once enabled, Live Unit Testing helps you quickly see whether the code you’re writing is covered and if the tests that cover it are passing without leaving the editor. Unit test results and coverage visualizations appear on a line-by-line basis in the code editor.

There are three potential states for any given line:

A line of executable code that is covered by at least one failing test is decorated with a red “x”.
A line of executable code that is covered by only passing tests is decorated with a green “✓”.
A line of executable code that is not covered by any test is decorated it with a blue dash “-”

Live Unit Testing coverage visualization is updated immediately as you modify code in the code editor. While processing the edits, visualization changes to indicate that the data is not up-to-date, as shown below. Once processing is done, it transitions to one of the final states described earlier:

Quickly navigate to failed test

At any point in time you can hover over the “✓” or “x” to see how many tests are hitting the given line, as seen in image below:

To see what tests are exercising the given line, click on “✓” or “x”.

When you hover over the failed test in the tool tip, it expands to provide additional info to give more insight into the failure:

To navigate directly to the failed test, click on it in the expanded UI.

Seamlessly debug failed test, edit and continue without having to restart

From the failed test, you can easily debug to the product code, make edits, and continue, without having to pause or stop Live Unit Testing. However, if you want Live Unit Testing to pause when you are debugging then there is an option in Tools/Options page, as shown in section 7 below, for Live Unit Testing to do that.

Include/Exclude targeted test methods or projects for large solutions

Live unit testing enables you to avoid the overhead of continuously running the unit tests for code that you aren’t touching or affecting, especially when you are dealing with a large code base. When you enable Live Unit Testing on a solution with 10 or more projects, you will see a dialog box, that gives a choice to include all or exclude all projects:

If you select Yes, you get coverage for all projects. If you select No, then Live Unit Testing excludes all projects from Live Unit Testing. You can then go ahead and include targeted tests, by right clicking a test project in solution explorer and choosing Live Tests/Include:

The smallest unit of inclusion in or exclusion from Live Unit Testing is a test method.

You can also include or exclude tests by right-clicking in the code editor:

  • If you right click anywhere inside the method then that method can be included or excluded.
  • If you right click outside the method body but somewhere inside the containing class, then all tests inside that class can be included or excluded.
  • If you right click outside the class but somewhere inside the file, then all tests in that file can be included or excluded.

The last include/exclude action always wins. If you had explicitly excluded some tests within a class and go back to it later and include the entire containing class, then all tests inside that class will get included. This includes the tests that were previously explicitly excluded.

Include/Exclude state is saved as a user setting and is remembered when a solution is closed and reopened.

See FAQs section below for more information on how to exclude tests.

Integrated with Test Explorer

The Test Explorer Window and Live Unit Testing are synchronized. As you change your code, Live Unit Testing runs the impacted tests, and only those Tests are shown bright in the Test Explorer Window. The non-impacted tests are dimmed out as shown in image below:

Configurable with Tools/Options/Settings

To configure the available options for Live Unit Testing go to Visual Studio’s Options from the Tools menu, and select Live Unit Testing in the left pane of the Options dialog. The configurable options include:

  • Whether Live Unit Testing runs automatically when a solution is opened.
  • Whether Live Unit Testing pauses when a solution is built and debugged, or when a system’s battery power falls below a specified threshold.
  • The interval after which a test case times out; the default is 30 seconds.
  • The number of test processes that Live Unit Testing would create.
  • The Live Unit Testing log information written to the output window. Options include no logging (None), error messages only (Error), error and informational messages (Info – this is the default), or all detail (Verbose).

Adaptable to work with three popular unit testing frameworks

Live Unit Testing in Visual Studio works with three popular unit testing frameworks: MSTest, xUnit.net, and NUnit. The adapters and frameworks referenced in every project in the solution must meet or exceed the minimum versions given below else Live Unit Testing will give an error. You can get these from NuGet.org.

Test FrameworkVisual Studio Adapter VersionFramework version
xUnit.netxunit.runner.visualstudio version 2.2.0-beta3-build1187xunit 2.0
NUnitNUnit3TestAdapter version 3.5.1NUnit version 3.5.0
MSTestMSTest.TestAdapter 1.1.4-preview [UPDATE 5/23/2017 : Minimum version needed is now updated to MSTest.TestAdapter 1.1.11]MSTest.TestFramework 1.0.5-preview [UPDATE 5/23/2017 : Minimum version needed is now updated to MSTest.TestFramework 1.1.11]

If you have older adapter and test framework references from your existing projects, be sure to remove them (make sure you remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework, if you are using MSTest.) and add the new ones for Live Unit Testing to work.

FAQs

You can find latest Live Unit Testing FAQs on our MSDN documentation wiki.

Q: Does Live Unit Testing work with .NET Core?

A: Yes. Live Unit Testing works with .NET Core and .NET Frameworks. Support for .NET Core was added recently in Visual Studio 2017 version 15.3 Preview.

Q: Why doesn’t Live Unit Testing work when I turn it on?

A: Output Window (when Live Unit Testing drop down is selected) should tell you why Live Unit Testing is not working. Live Unit testing may not work because of one of the following reasons:

  • If NuGet packages referenced by the projects in the solution have not been restored, then Live Unit Testing will not work. Doing an explicit build of the solution or restoring NuGet packages on the solution before turning Live Unit Testing on should resolve this issue.
  • If you are using MSTest-based tests in your projects, make sure that you remove the reference to Microsoft.VisualStudio.QualityTools.UnitTestFramework and add references to the latest MSTest NuGet packages (see above under the “Adaptable to work with three popular unit testing frameworks” section).
  • At least one project in your solution should have either a NuGet reference or direct reference to xUnit, NUnit or MSTest test framework. This project should also reference corresponding Visual Studio test adapters NuGet package. Visual Studio test adapter can also be referenced through a .runsettings file. The .runsettings file will need to have an entry like the one below:
Debug

Visual Studio Debugger Download

Microsoft visual studio for mac
<RunSettings>
<RunConfiguration>
<TestAdaptersPaths>c:PathToYourTestAdapter</TestAdaptersPaths>
</RunConfiguration>
</RunSettings>
Q:Can I customize my Live Unit Testing builds?

A: If your solution requires custom steps to build for instrumentation (Live Unit Testing) that are not required for the “regular” non-instrumented build, then you can add code to your project or .targets files that checks for the BuildingForLiveUnitTesting property and performs custom pre/post build steps. You can also choose to remove certain build steps (like publishing or generating packages) or to add build steps (like copying prerequisites) to a Live Unit Testing build based on this project property. This will not alter your regular build in any way and will only impact Live Unit Testing builds. For e.g., there may be a target which produces NuGet packages during a regular build. You probably do not want NuGet packages to be generated after every edit you make. So, you can disable that target in Live Unit Testing build by doing something like the following:

<Target Name='GenerateNuGetPackages' BeforeTargets='AfterBuild' Condition='$(BuildingForLiveUnitTesting)' != 'true'>
<Exec Command='$(MSBuildThisFileDirectory)..toolsGeneratePackages '/>
</Target>
Q: Why do I get the following error when Live Unit Testing tries to build my solution “… appears to unconditionally set ‘<OutputPath>’ or ‘<OutDir>’. Live Unit Testing will not execute tests from the output assembly”?

A: This can happen if the build process for your solution unconditionally overrides the <OutputPath> or <OutDir> such that it does not fall under <BaseOutputPath>. In such cases, Live Unit Testing will not work because it also overrides these to ensure that build artifacts are dropped to a folder under <BaseOutputPath>. If you must override the location where you want your build artifacts to be dropped in a regular build, then override the <OutputPath> conditionally and based on <BaseOutputPath>. For e.g. if your build is overriding the <OutputPath> as shown below:

<Project>
<PropertyGroup>
<OutputPath>$(SolutionDir)Artifacts$(Configuration)bin$(MSBuildProjectName)</OutputPath>
</PropertyGroup>
</Project>

Then you can replace it with the text below:

<Project>
<PropertyGroup>
<BaseOutputPath Condition='$(BaseOutputPath)' ''>$(SolutionDir)Artifacts$(Configuration)bin$(MSBuildProjectName)</BaseOutputPath>
<OutputPath Condition='$(OutputPath)' ''>$(BaseOutputPath)</OutputPath>
</PropertyGroup>
</Project>

This ensures that <OutputPath> lies within the <BaseOutputPath> folder.

Do not override <OutDir> directly in your build process; override <OutputPath> instead for build artifacts to be dropped to a specific location.

Q: I want the artifacts of a Live Unit Testing build to go to a specific location instead of the default location under the .vs folder. How can I change that?

A: Set the LiveUnitTesting_BuildRoot user level environment variable to the path where you want the Live Unit Testing build artifacts to be dropped.

Q: How is running tests from Test Explorer window different from running tests in Live Unit Testing?

A: There are several differences:

  • Running or debugging tests from the Test Explorer window runs regular binaries, whereas Live Unit Testing runs instrumented binaries. If you want to debug instrumented binaries, adding a Launch method call in your test method causes the debugger to launch whenever that method is executed (including when it is executed by Live Unit Testing), and you can then attach and debug the instrumented binary. However, our hope is that instrumentation is transparent to you for most user scenarios, and you do not need to debug instrumented binaries.
  • Live Unit Testing does not create a new AppDomain to run tests, but tests run from the Test Explorer window do create a new AppDomain.
  • Live Unit Testing runs tests in each test assembly sequentially, whereas if you run multiple tests from the Test Explorer window and you selected the Run Tests in Parallel button, they will run in parallel.
  • Discovery and execution of tests in Live Unit Testing goes through version 2 of TestPlatform, whereas the Test Explorer window still uses version 1. You should not notice a difference in most cases though.
  • Test Explorer currently runs tests in a single-threaded apartment (STA) by default, whereas Live Unit Testing runs tests in a multithreaded apartment (MTA). To have MSTest tests run in STA in Live Unit Testing, decorate the test method or the containing class with the <STATestMethod> or <STATestClass> attribute that can be found in the MSTest.STAExtensions 1.0.3-beta NuGet package. For NUnit, decorate the test method with the < RequiresThread(ApartmentState.STA)> attribute, and for xUnit, with the <STAFact> attribute.
Q: How do I exclude tests from participating in Live Unit Testing?

A: Please refer to include/exclude functionality mentioned above for user specific setting. This is extremely useful when you want to run a specific set of tests for a particular edit session or to persist your own personal preferences. For solution specific settings, you can apply the ExcludeFromCodeCoverageAttribute programmatically to exclude methods, properties, classes, or structures from getting instrumented by Live Unit Testing. Additionally, you can also set <ExcludeFromCodeCoverage> property to “true” in your project file, to exclude the whole project from getting instrumented. The tests which have not been instrumented will still be run by Live Unit Testing but coverage from those will not be visualized.

You can check if “Microsoft.CodeAnalysis.LiveUnitTesting.Runtime” is loaded in the current AppDomain and disable tests based on that. For e.g. you could do something like the following with xUnit:

[ExcludeFromCodeCoverage]
public class SkipLiveFactAttribute : FactAttribute
{
private static bool s_lutRuntimeLoaded = AppDomain.CurrentDomain.GetAssemblies().Any(a => a.GetName().Name 'Microsoft.CodeAnalysis.LiveUnitTesting.Runtime');
public override string Skip => s_lutRuntimeLoaded ? 'Test excluded from Live Unit Testing' : ';
}
public class Class1
{
[SkipLiveFact]
public void F()
{
Assert.True(true);
}
}
Q: Why are Win32 PE headers different in instrumented assemblies built by Live Unit testing?

A: There is a known bug that may result in Live Unit Testing builds failing to embed the following Win32 PE Header data. Tests that rely on these values may fail when executed by Live Unit testing:

  • File Version (specified by AssemblyFileVersionAttribute in code)
  • Win32 Icon (specified by /win32icon: on command line)
  • Win32 Manifest (specified by /win32manifest: on command line)
Q: Why does Live Unit testing keep building my solution all the time even if I am not making any edits?

A: This can happen if the build process of your solution generates code which are part of the solution itself, and your build target files do not have appropriate inputs and outputs specified. Targets should be given a list of inputs and outputs so that MSBuild can perform the appropriate up-to-date checks and determine whether a new build is required. Live Unit Testing will kick off a build whenever it detects that source files have changed. Because the build of your solution generates source files, Live Unit Testing will get into infinite build loop. If, however the inputs and outputs of the target are checked then when Live Unit Testing starts the second build (after detecting the newly generated source files from previous build), it will break out of the loop because the inputs and outputs checks will indicate that everything is up-to-date.

Q: How does Live Unit testing work with the Lightweight Solution Load feature?

A: Live Unit Testing currently doesn’t work well with the Lightweight Solution load feature if all projects in the solution are not yet loaded. You may get incorrect coverage information in such scenarios.

Q: Why does Live Unit Testing not capture coverage from a new process created by a test?

A: This is a known issue which we were not able to fix in Visual Studio 2017 version 15.0. It is now fixed in Visual Studio 2017 version 15.2.

Q: Why does nothing happen after I include or exclude tests from Live Test Set?

A: This is a known issue which we were not able to fix in Visual Studio 2017 version 15.0. It is now fixed in Visual Studio 2017 version 15.2.

Q: Why do I not see any icons in the editor even though Live Unit Testing seems to be running the tests based on the messages in the output window?

A: This will happen if the assemblies that Live Unit Testing is operating upon is not instrumented for any reason. For e.g. Live Unit Testing is not compatible with projects that set <UseHostCompilerIfAvailable>false</UseHostCompilerIfAvailable>. In this case, your build process will need to be updated to either not set this or change it to false for Live Unit Testing to work.

Q: How do I collect more detailed logs to file bug reports?

A: You can do several things to collect more detailed logs:

  • Go to Tools > Options > Live Unit Testing and change the logging option to verbose. This causes more detailed logs to be shown in the output window.
  • Set the LiveUnitTesting_BuildLog user environment variable to the path of the file you want to use to capture the MSBuild log. Detailed MSBuild log messages from Live Unit Testing builds can then be retrieved from this file.
  • Create a user-level environment variable named VS_UTE_DIAGNOSTICS and set it to 1 (or any value) and restart Visual Studio. Now you should see lots of logging in the Output – Tests tab in Visual Studio.

Conclusion

Live Unit Testing will improve your productivity, test coverage and software quality. .NET developers, check out this feature in Visual Studio 2017. For developers who are part of a team that practices test-driven development, Live Unit Testing gamifies their workflow; in other words, all their tests will fail at first, and as they implement each method, they will see them turn green. It will evoke the same feeling as a nod of approval from your coach, who is watching you intently from the sidelines, as you practice your art!

Watch this Live Unit Testing video, where we demonstrate this feature:

Joe Morris, Senior Program Manager, Visual Studio
@_jomorris

Joe has been with Microsoft for 19 years, with a particular focus on static analysis and developer productivity for the last three years.

Manish Jayaswal, Principal Engineering Manager, Visual Studio
@jayaswalmk

Manish has years of management experience in commercial software development with deep technical expertise in compiler technology, debugger, programming languages, quality assurance and engineering system.

Since it was released a little more than a year ago, Visual Studio 2017 for Mac has grown from being an IDE primarily focused on mobile application development using Xamarin to one that includes support for all major .NET cross-platform workloads including Xamarin, Unity, and .NET Core. Our aspiration with Visual Studio for Mac is to bring the Visual Studio experiences that developers have come to know and love on Windows to the MacOS and to provide an excellent IDE experience for all .NET cross-platform developers.

Over the past year, we added several new capabilities to Visual Studio for Mac including .NET Core 2; richer language services for editing JavaScript, TypeScript, and Razor pages; Azure Functions; and the ability to deploy and debug .NET Core apps inside Docker containers. At the same time, we have continued to improve Xamarin mobile development inside Visual Studio for Mac by adding same-day support for the latest iOS and Android SDKs, improving the visual designers and streamlining the emulator and SDK acquisition experiences. And we have updated the Unity game development experience to reduce launch times of Visual Studio for Mac when working together with the Unity IDE. Finally, we have been investing heavily in fundamentals such as customer feedback via the Report-a-Problem tool, accessibility improvements, and more regular updates of components that we share with the broader .NET ecosystem such as the .NET compiler service (“Roslyn”), and the .NET Core SDKs. We believe that these changes will allow us to significantly accelerate delivery of new experiences in the near future.

While we will continue to make improvements to Visual Studio 2017 for Mac into early next year, we also want to start talking about what’s next: Visual Studio 2019 for Mac. Today, we are publishing a roadmap for Visual Studio for Mac, and in this blog post, I wanted to write about some of the major themes of feedback we are hearing and our plans to address them as described in our roadmap.

Improving the performance and reliability of the code editor

Improving the typing performance and reliability is our single biggest focus area for Visual Studio 2019 for Mac. We plan to replace most of the internals of the Visual Studio for Mac editor with those from Visual Studio. Combined with the work to improve our integration of various language services, our aspiration is to bring similar levels of editor productivity from Visual Studio to Visual Studio for Mac. Finally, as a result of this work, we will also be able to address a top request from users to add Right-To-Left (RTL) support to our editor.

Supporting Team Foundation Version Control

Including support for Team Foundation Server, with both Team Foundation Version Control (TFVC) and Git as the source control mechanisms, has been one of the top requested experiences on the Mac. While we currently have an extension available for Visual Studio 2017 for Mac that adds support for TFVC, we will integrate it into the core of the source control experience in Visual Studio 2019 for Mac.

Increased productivity when working with your projects

Vs Code Debug Unit Tests

The C# editor in Visual Studio for Mac will be built on top of the same Roslyn backend used by Visual Studio on Windows and will see continuous improvements. In Visual Studio 2017 for Mac (version 7.7), we will enable the Roslyn-powered brace completion and indentation engine which helps improve your efficiency and productivity while writing C# code. We’re also making our quick fixes and code action more discoverable by introducing a light-bulb experience. With the light bulb, you’ll see recommendations highlighted inline in the editor as you code, with quick keyboard actions to preview and apply the recommendations. In the Visual Studio 2019 for Mac release, we’ll also dramatically reduce the time it takes you to connect to your source code and begin working with it in the product, by introducing a streamlined “open from version control” dialog with a brand-new Git-focused workflow.

.NET Core and ASP.NET Core support

In future updates to Visual Studio 2017 for Mac, we will add support for .NET Core 2.2. We will add the ability to publish ASP.NET Core projects to a folder. We will also add support for Azure Functions 2.0, as well as update the New Functions Project dialog to support updating to the latest version of Azure Functions tooling and templates. In Visual Studio 2019 for Mac, we will add support for .NET Core 3.0 when it becomes available in 2019. We will add more ASP.NET Core templates and template options to Visual Studio for Mac and improve the Azure publishing options. Finally, building upon the code editor changes described above, we will improve all our language services supporting ASP.NET Core development including Razor, JavaScript and TypeScript.

Xamarin support

In addition to continuing to make improvements to the Xamarin platform itself, we will focus on improving Android build performance and improving the reliability of deploying iOS and Android apps. We will make it easy to acquire the Android emulators from within the Visual Studio for Mac IDE. Finally, we aim to make further improvements in the Xamarin.Forms Previewer and the Xamarin.Android Designer as well as the XAML language service for Xamarin Forms.

Unity support

We continue to invest in improving the experience of game developers using Unity to write and debug cross platform games as well as 2D and 3D content using Visual Studio for Mac. Unity now supports a .NET 4.7 and .NET Standard 2.0 profile, and we’re making sure that Visual Studio for Mac works out of the box to support those scenarios. Unity 2018.3 ships with Roslyn, the same C# compiler that is used with Visual Studio for Mac, and we’re enabling this for your IDE. In addition to this, we’ll be bringing our fine-tuned Unity debugger from the Visual Studio Tools for Unity to Visual Studio for Mac for a more reliable and faster Unity debugging experience.

Visual Studio For Mac Download

Help us shape Visual Studio 2019 for Mac!

By supporting installation of both versions of the product side-by-side, we’ll make it easy for you to try out the Visual Studio 2019 for Mac preview releases while we are still also working on the stable Visual Studio 2017 for Mac releases in parallel.

We don’t have preview bits to share with you just yet, but we wanted to share our plans early so you can help us shape the product with your feedback that you can share through our Developer Community website. We will update our roadmap for Visual Studio for Mac once a quarter to reflect any significant changes. We will also post an update to our roadmap for Visual Studio soon.