Responsible use of GitHub Copilot Chat in GitHub

About GitHub Copilot Chat in GitHub

GitHub Copilot Chat in GitHub is a chat interface that lets you interact with GitHub Copilot, to ask and receive answers to coding-related questions within GitHub.

The chat interface provides access to coding information and support without requiring you to navigate documentation or search online forums.

GitHub Copilot Chat can answer a wide range of coding-related questions on topics including syntax, programming concepts, test cases, debugging, and more. GitHub Copilot Chat is not designed to answer non-coding questions or provide general information on topics outside of coding.

The primary supported language for Copilot Chat in GitHub is English.

GitHub Copilot Chat works by using a combination of natural language processing and machine learning to understand your question and provide you with an answer. This process can be broken down into a number of steps.

Input processing

The input prompt from the user is pre-processed by the Copilot Chat system, combined with contextual information (for example, the current date and time and the name of the repository the user is currently viewing), and sent to a large language model. User input can take the form of code snippets or plain language.

The large language model will take the prompt, gather additional context (for example repository data stored on GitHub or search results from Bing), and provide a response based on the prompt. The system is only intended to respond to coding-related questions.

Language model analysis

The pre-processed prompt is then passed through the Copilot Chat language model, which is a neural network that has been trained on a large body of text data. The language model analyzes the input prompt.

Response generation

The language model generates a response based on its analysis of the input prompt and the context provided to it. The language model can gather additional context (for example repository data stored on GitHub or search results from Bing), and provide a response based on the prompt. This response can take the form of generated code, code suggestions, or explanations of existing code.

Output formatting

The response generated by Copilot Chat is formatted and presented to the user. Copilot Chat may use syntax highlighting, indentation, and other formatting features to add clarity to the generated response. Depending upon the type of question from the user, links to context that the model used when generating a response, such as source code files, issues, Bing search results, or documentation, may also be provided.

Copilot Chat is intended to provide you with the most relevant answer to your question. However, it may not always provide the answer you are looking for. Users of Copilot Chat are responsible for reviewing and validating responses generated by the system to ensure they are accurate and appropriate. Additionally, as part of our product development process, we undertake red teaming to understand and improve the safety of Copilot Chat. Input prompts and output completions are run through content filters. The content filtering system detects and prevents the output on specific categories of content including harmful, offensive, or off-topic content. For more information on improving the performance of Copilot Chat, see Improving performance for Copilot Chat.

Use cases for Copilot Chat

Copilot Chat can provide coding assistance in a variety of scenarios.

Answering coding questions

You can ask Copilot Chat for help or clarification on specific coding problems and receive responses in natural language format or in code snippet format.

The response generated by Copilot Chat may use the model's training data set, search results from Bing, code in your repositories, and Markdown documentation in your knowledge bases to answer your questions.

This can be a useful tool for programmers, as it can provide guidance and support for common coding tasks and challenges.

Explaining code and suggesting improvements

Copilot Chat can help explain selected code by generating natural language descriptions of the code's functionality and purpose. This can be useful if you want to understand the code's behavior or for non-technical stakeholders who need to understand how the code works. For example, if you select a function or code block in the code editor, Copilot Chat can generate a natural language description of what the code does and how it fits into the overall system. This can include information such as the function's input and output parameters, its dependencies, and its purpose in the larger application.

Copilot Chat can also suggest potential improvements to selected code, such as improved handling of errors and edge cases, or changes to the logical flow to make the code more readable.

By generating explanations and suggesting related documentation, Copilot Chat may help you to understand the selected code, leading to improved collaboration and more effective software development. However, it's important to note that the generated explanations and documentation may not always be accurate or complete, so you'll need to review, and occasionally correct, Copilot Chat's output.

Proposing code fixes

Copilot Chat can propose a fix for bugs in your code by suggesting code snippets and solutions based on the context of the error or issue. This can be useful if you are struggling to identify the root cause of a bug or you need guidance on the best way to fix it. For example, if your code produces an error message or warning, Copilot Chat can suggest possible fixes based on the error message, the code's syntax, and the surrounding code.

Copilot Chat can suggest changes to variables, control structures, or function calls that might resolve the issue and generate code snippets that can be incorporated into the codebase. However, it's important to note that the suggested fixes may not always be optimal or complete, so you'll need to review and test the suggestions.

Planning coding tasks

Copilot Chat can read a GitHub issue and summarize it, answer questions about it, or propose next steps. This can be useful if you have a long, complex issue with many comments, and you want to understand it quickly or figure out what to do next.

However, it's important to note that Copilot Chat's answers and summaries may not always be accurate or complete, so you'll need to review Copilot Chat's output for accuracy.

Finding out about releases, discussions, and commits

Copilot Chat can help you find out what changed in a specific release, it can summarize the information in a discussion, and it can explain the changes in a specific commit. This can be useful if, for example, you are new to a project, you want to quickly get the gist of a discussion, or you need to work on code that someone else wrote. However, it's important to note that Copilot Chat's summaries of releases, discussions, and commits may not always be accurate or complete.

Improving performance for Copilot Chat

Copilot Chat can support a wide range of practical applications like Q&A, code generation, code analysis, and code fixes, each with different performance metrics and mitigation strategies. To enhance performance and address some of the limitations of Copilot Chat, there are various measures that you can adopt. For more information on the limitations of Copilot Chat, see "Limitations of GitHub Copilot Chat."

Keep your prompts on topic

Copilot Chat is intended to address queries related to coding exclusively. Therefore, limiting the prompt to coding questions or tasks can enhance the model's output quality.

Use Copilot Chat as a tool, not a replacement

While Copilot Chat can be a powerful tool for generating code, it is important to use it as a tool rather than a replacement for human programming. You should always review and test the code generated by Copilot Chat to ensure that it meets your requirements and is free of errors or security concerns.

Use secure coding and code review practices

While Copilot Chat can generate syntactically correct code, it may not always be secure. You should always follow best practices for secure coding, such as avoiding hard-coded passwords or SQL injection vulnerabilities, as well as following code review best practices, to address Copilot Chat's limitations.

Provide feedback

If you encounter any issues or limitations with Copilot Chat in GitHub, we recommend that you provide feedback by clicking the thumbs down icon below each chat response. This can help the developers to improve the tool and address any concerns or limitations.

Stay up to date

Copilot Chat is a new technology and is likely to evolve over time. For GitHub Copilot Chat in GitHub you will always have access to the latest product experience. You should stay up to date with any new security risks or best practices that may emerge.

Limitations of GitHub Copilot Chat

Depending on factors such as your codebase and input data, you may experience different levels of performance when using Copilot Chat. The following information is designed to help you understand system limitations and key concepts about performance as they apply to Copilot Chat.

Limited scope

Copilot Chat has been trained on a large body of code but still has a limited scope and may not be able to handle more complex code structures or obscure programming languages. For each language, the quality of suggestions you receive may depend on the volume and diversity of training data for that language. For example, JavaScript is well-represented in public repositories and is one of GitHub Copilot's best supported languages. Languages with less representation in public repositories may be more challenging for Copilot Chat to provide assistance with. Additionally, Copilot Chat can only suggest code based on the context of the code being written, so it may not be able to identify larger design or architectural issues.

Potential biases

Copilot's training data (drawn from existing code repositories) and context gathered by the large language model (for example, Bing search results) may contain biases and errors that can be perpetuated by the tool. Additionally, Copilot Chat may be biased towards certain programming languages or coding styles, which can lead to suboptimal or incomplete code suggestions.

Security risks

Copilot Chat generates code based on the context of the code being written, which can potentially expose sensitive information or vulnerabilities if not used carefully. You should be careful when using Copilot Chat to generate code for security-sensitive applications and always review and test the generated code thoroughly.

Matches with public code

Copilot Chat is capable of generating new code, which it does in a probabilistic way. While the probability that it may produce code that matches code in the training set is low, a Copilot Chat suggestion may contain some code snippets that match code in the training set.

If you have disabled suggestions that match public code then Copilot Chat utilizes filters that prevent it from showing code that matches code found in public repositories on GitHub. However, you should always take the same precautions as you would with any code you write that uses material you did not independently originate, including precautions to ensure its suitability. These include rigorous testing, IP scanning, and checking for security vulnerabilities.

If you have enabled suggestions that match public code then Copilot Chat displays a message if matching code is found. The message includes links to repositories on GitHub that contain matching code, and any license details that were found. For more information, see Finding public code that matches GitHub Copilot suggestions.

Inaccurate code

One of the limitations of Copilot Chat is that it may generate code that appears to be valid but may not actually be semantically or syntactically correct or may not accurately reflect the intent of the developer. To mitigate the risk of inaccurate code, you should carefully review and test the generated code, particularly when dealing with critical or sensitive applications. You should also ensure that the generated code adheres to best practices and design patterns and fits within the overall architecture and style of the codebase.

Inaccurate responses to non-coding topics

Copilot Chat is not designed to answer non-coding questions, and therefore its responses may not always be accurate or helpful in these contexts. If a user asks Copilot Chat a non-coding question, it may generate an answer that is irrelevant or nonsensical, or it may simply indicate that it is unable to provide a useful response.

Leveraging a web search to answer a question

Depending on the question you ask, GitHub Copilot Chat can optionally use a Bing search to help answer your question. Copilot will use Bing for queries about recent events, new trends or technologies, highly specific subjects, or when a web search is explicitly requested by the user. Your GitHub Enterprise administrator can enable Bing for your whole enterprise, or can delegate this decision to the organizational administrator. For more information, see Managing policies and features for Copilot in your enterprise.

When leveraging Bing, Copilot will use the content of your prompt, as well as additional available context, to generate a Bing search query on your behalf that is sent to the Bing Search API. Copilot will provide a link to the search results with its response. The search query sent to Bing is governed by Microsoft's Privacy Statement.

Next steps

For details of how to use GitHub Copilot Chat in GitHub, see:

• Asking GitHub Copilot questions in GitHub in the GitHub Enterprise Cloud documentation.

Further reading

• GitHub Pre-release License Terms
• GitHub Copilot Trust Center