Alvin Chardon
Software Design Engineer
Microsoft Corporation
March 2005
Summary: For several years Visual C++ has been an outstanding product that offers a conformant C/C++ compiler, a superior debugger, and a feature-rich integrated development environment (IDE). This article focuses on a specific feature of our IDE: IntelliSense. Learn how the feature was implemented, how it works, and what to expect from its functionality both now and in the future. (5 printed pages)
Introduction
The Visual C++ Approach to IntelliSense
Implementation Limits
Past, Present, and Future of IntelliSense in Visual C++
What's New in Visual Studio 2005
IntelliSense is the term used to identify the set of dynamic user code information features of Visual C++ such as Quick Information, Auto Completion, and Parameter Help.
Quick Information (QI) gives you information about the identifier in question, such as variable type, type definition, function signature, and so on. Users request QI by positioning the cursor or mouse pointer on top of any identifier in a C++ project; the QI information will be displayed in a yellow pop-up box.
Auto Completion (ACP) gives you information about the members of a class, such as fields, methods, properties, events, and so on. Users request ACP by typing a member selection operator ('.' or '->') or the scope resolution operator ('::') after an object or pointer or a namespace or class name respectively.
Parameter Help (PH) gives you information about a function call or a template instantiation, such as function parameters and overloads, template parameters, and so on. Users request PH by typing the opening parenthesis ('(') or angle bracket ('<') after a function name or template name respectively.
After reading this document, readers will have a better understanding of how the feature was designed and implemented and what functionality to expect.
In order to provide dynamic information about user code, two key pieces of information are needed: the language the code is written in and the code that is being written. For this, we needed a C++ parser that could be loaded dynamically by the IDE every time the user code changes. Since we already have a C++ parser in our compiler, we chose to reuse the existing parser and modify it for the task at hand.
This approach has many advantages as well as some potential drawbacks; the more notable advantages being that we get a full C++ parser for free and that both the language support in the compiler and the IntelliSense support in the IDE can be kept at par; that is, when a new feature is added to the language, we automatically get IntelliSense support for it.
On the other hand, it is very common that the user code at the design and coding stage is not legal C++ code yet. Keeping in mind that the user code at this stage is a main product scenario for an IDE, our C++ compiler parser had to be adapted to not only understand and retrieve information about legal C++ code, but to be able to retrieve this information from not yet legal C++ code. Whereas our compiler parser can simply alert the user about syntactic and semantic mistakes and then gracefully exit, our IntelliSense parser must make assumptions about and guess what the illegal C++ user code might mean and continue parsing and retrieving information. This means that in our implementation, we might not always retrieve or display the correct information when our user code is in illegal compilation states. The more malformed the code is, the less likely it will be that the customer will get accurate information. There are some cases in which IntelliSense information is not 100-percent accurate for well-formed code too. See Implementation Limits for more details.
Several components are involved in the process of gathering, requesting and displaying IntelliSense information. These include the user code, the IntelliSense parser, and the Visual C++ IDE. When a C++ project is opened, the IDE will load the IntelliSense parser to retrieve all the IntelliSense information from the user code and create a solution-specific database file in which to store the information. (The process of parsing user code, creating a database—called NCB store—and storing all the IntelliSense information extracted from the user code is called population.) The compilation options for the parser are obtained from the project system and all information is displayed through the environment user interface. Every time information is requested by the user through the UI, the IDE queries the database file using the IntelliSense parser. See Figure 1 for a visual explanation of the communication process between the user's source code, the IDE, and the IntelliSense parser.
ISO/ANSI C++ is a complex language in itself. Microsoft Visual C++ added Managed Extensions for C++ and now our new C++/CLI syntax, thus adding extra complexity to our language and two new compilation modes for a total of three: native code, managed code using Managed Extensions, and managed code using the new syntax. This makes our IntelliSense parser code more complex in auto-recovering since we have three valid syntaxes under three different compilation modes. Comparing the Microsoft Visual C++ language against other Microsoft languages such as Visual C# and Visual Basic we observe a great difference in language complexity. Not only the is the syntax itself more complex and varied, but both these other languages are managed only and thus other mechanisms for retrieving information about user code exist as part of the .NET Framework. Visual C++ as a pioneer in the field has greatly contributed to the development of these other technologies, but still relies on our initial implementation approach of reusing our compiler parser, and thus is somewhat limited.
Apart from the limited error recovery mechanism already mentioned, here is a list of scenarios where customers might expect some problems with IntelliSense.
Common problems
The most common problem that a user will face with IntelliSense is that no information is displayed for a given scope, file, or project. In most cases it will fall under one of the following circumstances:
In some of these cases, bad side effects like memory corruption or an application hang may occur. As part of our security work in Visual Studio 2005, we have modified our parser to skip parsing when extremely malformed user code is seen, thus mitigating to some extent the risk of unrecoverable errors. Note that an analysis of all compiler errors, specifically fatal errors, has been made to decide which ones are safe to continue the parse and which ones are not.
This means that having code with multiple scopes on the same line can cause IntelliSense requests to fail on those scopes.
In most cases we will parse the header file only once (for speed reasons), even if the same header file is included under different macro states in the same source file, on different source files in the same project, or in different projects in the same solution. This means that the files that depend on this header file in a macro state that is not the one populated may not show 100-percent accurate IntelliSense information.
We now dynamically populate all referenced native libraries (Microsoft Foundation Classes [MFC], Active Template Library [ATL], C Runtime Libraries [CRT], and Standard Template Library [STL]), managed assemblies, and imported modules in a project. The slight slowness is due to the one-time population of all information from these headers and modules into the NCB store. The performance hit will be experienced only when first creating an NCB store for a project and depends on the project's size. This might be more noticeable when writing managed code that references many assemblies. In previous versions of Visual Studio, we shipped pre-built NCB stores containing the IntelliSense information from all these libraries. Switching these models, we gained the benefit of faster queries and more accurate information in a project's NCB store while trading off this one hit.
As pioneers in the field of IntelliSense for Microsoft, not only have we helped push forward IntelliSense technologies for other Microsoft languages but we have evolved our own technologies across the different shipped products.
We changed our model of shipping pre-built NCB stores for our libraries to dynamically parsing all header files and modules referenced in user code and increased performance. Also, we have come from limited template support to supporting nested templates, partial and explicit specializations, and now even managed templates and generics (the Common Language Runtime's version of templates). In Visual Studio 2005, we have implemented almost all of our language (see Implementation Limits for more details), including support for the newly introduced syntax for writing managed applications, which is almost a totally new language in itself.
We will continue to innovate in the IntelliSense field in future products; there is still room for improvement in the area. Visual Studio 2005 will help improve the productivity and efficiency of our developers. Here are some of the new, diverse features that enrich the coding experience in Visual Studio 2005:
Microsoft Visual C++ will always be committed to our customer feedback and to bringing the best experience to developers around the World.
I would like to thank my colleagues in the Visual C++ Compiler and IDE teams for their great work and their guidance in compiling the information necessary to put together this article. Thanks go to Tanveer Gani, Ameya Limaye, Tarek Madkour, and Li Shao. Thanks for sharing your feedback and expertise. Special thanks go to Fernmarie Rodriguez for her unconditional love and support. My heart and thoughts will always be with you.