diff --git a/README.md b/README.md index 64b68a37a..6f1cda8fa 100644 --- a/README.md +++ b/README.md @@ -80,6 +80,7 @@ Alternatively, to manually configure VS Code, choose the appropriate JSON block ### Install in other MCP hosts + - **[GitHub Copilot in other IDEs](/docs/installation-guides/install-other-copilot-ides.md)** - Installation for JetBrains, Visual Studio, Eclipse, and Xcode with GitHub Copilot - **[Claude Applications](/docs/installation-guides/install-claude.md)** - Installation guide for Claude Desktop and Claude Code CLI - **[Codex](/docs/installation-guides/install-codex.md)** - Installation guide for Open AI Codex @@ -104,6 +105,7 @@ When no toolsets are specified, [default toolsets](#default-toolset) are used. GitHub Enterprise Cloud can also make use of the remote server. Example for `https://octocorp.ghe.com` with GitHub PAT token: + ``` { ... @@ -140,24 +142,30 @@ The MCP server can use many of the GitHub APIs, so enable the permissions that y
Handling PATs Securely ### Environment Variables (Recommended) + To keep your GitHub PAT secure and reusable across different MCP hosts: 1. **Store your PAT in environment variables** + ```bash export GITHUB_PAT=your_token_here ``` + Or create a `.env` file: + ```env GITHUB_PAT=your_token_here ``` 2. **Protect your `.env` file** + ```bash # Add to .gitignore to prevent accidental commits echo ".env" >> .gitignore ``` 3. **Reference the token in configurations** + ```bash # CLI usage claude mcp update github -e GITHUB_PERSONAL_ACCESS_TOKEN=$GITHUB_PAT @@ -180,6 +188,7 @@ To keep your GitHub PAT secure and reusable across different MCP hosts: - **Regular rotation**: Update tokens periodically - **Never commit**: Keep tokens out of version control - **File permissions**: Restrict access to config files containing tokens + ```bash chmod 600 ~/.your-app/config.json ``` @@ -193,6 +202,7 @@ the hostname for GitHub Enterprise Server or GitHub Enterprise Cloud with data r - For GitHub Enterprise Server, prefix the hostname with the `https://` URI scheme, as it otherwise defaults to `http://`, which GitHub Enterprise Server does not support. - For GitHub Enterprise Cloud with data residency, use `https://YOURSUBDOMAIN.ghe.com` as the hostname. + ``` json "github": { "command": "docker", @@ -328,6 +338,18 @@ If you don't have Docker, you can use `go build` to build the binary in the } ``` +### CLI utilities + +The `github-mcp-server` binary includes a few CLI subcommands that are helpful for debugging and exploring the server. + +- `github-mcp-server tool-search ""` searches tools by name, description, and input parameter names. Use `--max-results` to return more matches. +Example: + +```bash +docker run -i --rm ghcr.io/github/github-mcp-server tool-search "issue" --max-results 5 +github-mcp-server tool-search "issue" --max-results 5 +``` + ## Tool Configuration The GitHub MCP Server supports enabling or disabling specific groups of functionalities via the `--toolsets` flag. This allows you to control which GitHub API capabilities are available to your AI tools. Enabling only the toolsets that you need can help the LLM with tool choice and reduce the context size. @@ -349,6 +371,7 @@ To specify toolsets you want available to the LLM, you can pass an allow-list in ``` 2. **Using Environment Variable**: + ```bash GITHUB_TOOLSETS="repos,issues,pull_requests,actions,code_security" ./github-mcp-server ``` @@ -366,23 +389,29 @@ You can also configure specific tools using the `--tools` flag. Tools can be use ``` 2. **Using Environment Variable**: + ```bash GITHUB_TOOLS="get_file_contents,issue_read,create_pull_request" ./github-mcp-server ``` 3. **Combining with Toolsets** (additive): + ```bash github-mcp-server --toolsets repos,issues --tools get_gist ``` + This registers all tools from `repos` and `issues` toolsets, plus `get_gist`. 4. **Combining with Dynamic Toolsets** (additive): + ```bash github-mcp-server --tools get_file_contents --dynamic-toolsets ``` + This registers `get_file_contents` plus the dynamic toolset tools (`enable_toolset`, `list_available_toolsets`, `get_toolset_tools`). **Important Notes:** + - Tools, toolsets, and dynamic toolsets can all be used together - Read-only mode takes priority: write tools are skipped if `--read-only` is set, even if explicitly requested via `--tools` - Tool names must match exactly (e.g., `get_file_contents`, not `getFileContents`). Invalid tool names will cause the server to fail at startup with an error message @@ -435,9 +464,11 @@ GITHUB_TOOLSETS="all" ./github-mcp-server ``` #### "default" toolset + The default toolset `default` is the configuration that gets passed to the server if no toolsets are specified. The default configuration is: + - context - repos - issues @@ -1371,12 +1402,12 @@ The following sets of tools are available: Copilot -- **create_pull_request_with_copilot** - Perform task with GitHub Copilot coding agent - - `owner`: Repository owner. You can guess the owner, but confirm it with the user before proceeding. (string, required) - - `repo`: Repository name. You can guess the repository name, but confirm it with the user before proceeding. (string, required) - - `problem_statement`: Detailed description of the task to be performed (e.g., 'Implement a feature that does X', 'Fix bug Y', etc.) (string, required) - - `title`: Title for the pull request that will be created (string, required) - - `base_ref`: Git reference (e.g., branch) that the agent will start its work from. If not specified, defaults to the repository's default branch (string, optional) +- **create_pull_request_with_copilot** - Perform task with GitHub Copilot coding agent + - `owner`: Repository owner. You can guess the owner, but confirm it with the user before proceeding. (string, required) + - `repo`: Repository name. You can guess the repository name, but confirm it with the user before proceeding. (string, required) + - `problem_statement`: Detailed description of the task to be performed (e.g., 'Implement a feature that does X', 'Fix bug Y', etc.) (string, required) + - `title`: Title for the pull request that will be created (string, required) + - `base_ref`: Git reference (e.g., branch) that the agent will start its work from. If not specified, defaults to the repository's default branch (string, optional)
@@ -1384,19 +1415,21 @@ The following sets of tools are available: Copilot Spaces -- **get_copilot_space** - Get Copilot Space - - `owner`: The owner of the space. (string, required) - - `name`: The name of the space. (string, required) +- **get_copilot_space** - Get Copilot Space + - `owner`: The owner of the space. (string, required) + - `name`: The name of the space. (string, required) + +- **list_copilot_spaces** - List Copilot Spaces -- **list_copilot_spaces** - List Copilot Spaces
GitHub Support Docs Search -- **github_support_docs_search** - Retrieve documentation relevant to answer GitHub product and support questions. Support topics include: GitHub Actions Workflows, Authentication, GitHub Support Inquiries, Pull Request Practices, Repository Maintenance, GitHub Pages, GitHub Packages, GitHub Discussions, Copilot Spaces - - `query`: Input from the user about the question they need answered. This is the latest raw unedited user message. You should ALWAYS leave the user message as it is, you should never modify it. (string, required) +- **github_support_docs_search** - Retrieve documentation relevant to answer GitHub product and support questions. Support topics include: GitHub Actions Workflows, Authentication, GitHub Support Inquiries, Pull Request Practices, Repository Maintenance, GitHub Pages, GitHub Packages, GitHub Discussions, Copilot Spaces + - `query`: Input from the user about the question they need answered. This is the latest raw unedited user message. You should ALWAYS leave the user message as it is, you should never modify it. (string, required) +
## Dynamic Tool Discovery diff --git a/cmd/github-mcp-server/main.go b/cmd/github-mcp-server/main.go index af59ee0e6..27bffaff8 100644 --- a/cmd/github-mcp-server/main.go +++ b/cmd/github-mcp-server/main.go @@ -136,7 +136,6 @@ func initConfig() { viper.SetEnvPrefix("github") viper.SetEnvKeyReplacer(strings.NewReplacer("-", "_")) viper.AutomaticEnv() - } func main() { diff --git a/go.mod b/go.mod index 0bb6b2ae3..e50f52c72 100644 --- a/go.mod +++ b/go.mod @@ -3,6 +3,7 @@ module github.com/github/github-mcp-server go 1.24.0 require ( + github.com/fatih/color v1.18.0 github.com/google/go-github/v79 v79.0.0 github.com/google/jsonschema-go v0.4.2 github.com/josephburnett/jd v1.9.2 @@ -20,6 +21,8 @@ require ( github.com/gorilla/css v1.0.1 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/mailru/easyjson v0.7.7 // indirect + github.com/mattn/go-colorable v0.1.13 // indirect + github.com/mattn/go-isatty v0.0.20 // indirect github.com/stretchr/objx v0.5.2 // indirect github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82 // indirect go.yaml.in/yaml/v3 v3.0.4 // indirect @@ -34,6 +37,7 @@ require ( github.com/go-viper/mapstructure/v2 v2.5.0 github.com/google/go-querystring v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/lithammer/fuzzysearch v1.1.8 github.com/modelcontextprotocol/go-sdk v1.2.0 github.com/pelletier/go-toml/v2 v2.2.4 // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect diff --git a/go.sum b/go.sum index 94f41fa21..89c8b1dda 100644 --- a/go.sum +++ b/go.sum @@ -6,6 +6,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM= +github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU= github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8= github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0= github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k= @@ -44,11 +46,18 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4= +github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= +github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= +github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= +github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/microcosm-cc/bluemonday v1.0.27 h1:MpEUotklkwCSLeH+Qdx1VJgNqLlpY2KXwXFM08ygZfk= github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB5OBs92Dimizgw2cA= github.com/modelcontextprotocol/go-sdk v1.2.0 h1:Y23co09300CEk8iZ/tMxIX1dVmKZkzoSBZOpJwUnc/s= @@ -96,20 +105,53 @@ github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zI github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4= github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82 h1:BHyfKlQyqbsFN5p3IfnEUduWvb9is428/nNb5L3U01M= github.com/yudai/golcs v0.0.0-20170316035057-ecda9a501e82/go.mod h1:lgjkn3NuSvDfVJdfcVVdX+jpBxNmX4rDAzaS45IcYoM= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8= golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= +golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8= golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8= golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI= golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik= golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= +golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng= golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= +golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU= golang.org/x/tools v0.35.0 h1:mBffYraMEf7aa0sB+NuKnuCy8qI/9Bughn8dC2Gu5r0= golang.org/x/tools v0.35.0/go.mod h1:NKdj5HkL/73byiZSJjqJgKn3ep7KjFkBOkR/Hps3VPw= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= diff --git a/internal/ghmcp/server.go b/internal/ghmcp/server.go index 6090063f1..9e7dafcae 100644 --- a/internal/ghmcp/server.go +++ b/internal/ghmcp/server.go @@ -215,7 +215,7 @@ func NewMCPServer(cfg MCPServerConfig) (*mcp.Server, error) { cfg.Translator, github.FeatureFlags{ LockdownMode: cfg.LockdownMode, - InsiderMode: cfg.InsiderMode, + InsiderMode: cfg.InsiderMode, }, cfg.ContentWindowSize, featureChecker, @@ -235,7 +235,7 @@ func NewMCPServer(cfg MCPServerConfig) (*mcp.Server, error) { WithToolsets(enabledToolsets). WithTools(cfg.EnabledTools). WithFeatureChecker(featureChecker) - + // Apply token scope filtering if scopes are known (for PAT filtering) if cfg.TokenScopes != nil { inventoryBuilder = inventoryBuilder.WithFilter(github.CreateToolScopeFilter(cfg.TokenScopes)) diff --git a/pkg/tooldiscovery/search.go b/pkg/tooldiscovery/search.go new file mode 100644 index 000000000..e7adc029b --- /dev/null +++ b/pkg/tooldiscovery/search.go @@ -0,0 +1,314 @@ +package tooldiscovery + +import ( + "sort" + "strings" + + "github.com/google/jsonschema-go/jsonschema" + "github.com/lithammer/fuzzysearch/fuzzy" + "github.com/modelcontextprotocol/go-sdk/mcp" +) + +type SearchResult struct { + Tool mcp.Tool `json:"tool"` + Score float64 `json:"score"` + MatchedIn []string `json:"matchedIn"` // Signals that contributed to scoring (e.g. name:token, description, parameter:token). +} + +const ( + DefaultMaxSearchResults = 3 + + // Scoring weights used by scoreTool. + substringMatchScore = 5 + exactTokensMatchScore = 2.5 + descriptionMatchScore = 2 + prefixMatchScore = 1.5 + parameterMatchScore = 1 +) + +// SearchOptions configures search behavior. +type SearchOptions struct { + MaxResults int `json:"maxResults"` // Maximum number of results to return (default: 3) +} + +// Search returns the most relevant tools for a free-text query. +// +// Prefer using SearchTools and passing an explicit tool list. This function is +// kept for API compatibility and currently searches an empty tool set. +func Search(query string, options ...SearchOptions) ([]SearchResult, error) { + return SearchTools(nil, query, options...) +} + +// SearchTools is like Search, but searches across the provided tool list. +// +// Matching uses a weighted combination of: +// - tool name matches (strongest) +// - description matches +// - input parameter name matches (JSON schema property names) +// - fuzzy similarity as a tie-breaker +// +// Empty or whitespace-only queries return (nil, nil). +func SearchTools(tools []mcp.Tool, query string, options ...SearchOptions) ([]SearchResult, error) { + maxResults := getMaxResults(options) + + query = strings.TrimSpace(query) + if query == "" { + return nil, nil + } + + queryLower := strings.ToLower(query) + queryTokens := strings.Fields(queryLower) + normalizedQueryCompact := strings.ReplaceAll(strings.ReplaceAll(queryLower, " ", ""), "_", "") + + results := make([]SearchResult, 0, len(tools)) + for _, tool := range tools { + score, matchedIn := scoreTool(tool, queryLower, queryTokens, normalizedQueryCompact) + results = append(results, SearchResult{ + Tool: tool, + Score: score, + MatchedIn: matchedIn, + }) + } + + sort.Slice(results, func(i, j int) bool { return results[i].Score > results[j].Score }) + + // Filter out low-relevance results + const minScore = 1.0 + filtered := results[:0] + for _, r := range results { + if r.Score > minScore { + filtered = append(filtered, r) + } + } + results = filtered + + // Limit results + if len(results) > maxResults { + results = results[:maxResults] + } + + return results, nil +} + +// scoreTool assigns a relevance score to a tool for the given query. +// +// It combines several signals (substrings, token coverage, and similarity) from: +// - tool name +// - tool description +// - input parameter names (schema property names) +// +// MatchedIn records which signals contributed to the score for debugging/tuning. +func scoreTool( + tool mcp.Tool, + queryLower string, + queryTokens []string, + normalizedQueryCompact string, +) (score float64, matchedIn []string) { + nameLower := strings.ToLower(tool.Name) + descLower := strings.ToLower(tool.Description) + + normalizedNameCompact := strings.ReplaceAll(nameLower, "_", "") + nameTokens := splitTokens(nameLower) + propertyNames := lowerInputPropertyNames(tool.InputSchema) + + matches := newMatchTracker(3) + score = 0.0 + + // Strong boosts for direct substring matches + if strings.Contains(nameLower, queryLower) { + score += substringMatchScore + matches.Add("name:substring") + } + if strings.HasPrefix(nameLower, queryLower) { + score += prefixMatchScore + matches.Add("name:prefix") + } + if normalizedNameCompact == normalizedQueryCompact && len(queryTokens) > 1 { + score += exactTokensMatchScore + matches.Add("name:exact-tokens") + } + if strings.Contains(descLower, queryLower) { + score += descriptionMatchScore + matches.Add("description") + } + + for _, prop := range propertyNames { + if strings.Contains(prop, queryLower) { + score += parameterMatchScore + matches.Add("parameter") + } + } + + matchedTokens := make(map[string]struct{}) + + // Token-level matches for multi-word queries + for _, token := range queryTokens { + if strings.Contains(nameLower, token) { + score++ + matchedTokens[token] = struct{}{} + matches.Add("name:token") + } else if strings.Contains(descLower, token) { + score += 0.6 + matchedTokens[token] = struct{}{} + matches.Add("description:token") + } + + for _, prop := range propertyNames { + if strings.Contains(prop, token) { + // Only credit the first parameter match per token to avoid double-counting + score += 0.4 + matchedTokens[token] = struct{}{} + matches.Add("parameter:token") + break + } + } + } + + tokenCoverage := float64(len(matchedTokens)) + score += tokenCoverage * 0.8 + if len(queryTokens) > 1 && len(matchedTokens) == len(queryTokens) { + score += 2 // bonus when all tokens are matched somewhere + } + + // Prefer names that cover query tokens directly, with fewer extra tokens + nameTokenMatches := 0 + for _, qt := range queryTokens { + for _, nt := range nameTokens { + if strings.Contains(nt, qt) { + nameTokenMatches++ + break + } + } + } + if nameTokenMatches == len(queryTokens) { + score += 4.0 // all tokens present in name tokens + if len(nameTokens) == len(queryTokens) { + score += 2.0 // exact token count match (e.g., issue_write vs sub_issue_write) + } + } + extraTokens := len(nameTokens) - nameTokenMatches + if extraTokens > 0 { + score -= float64(extraTokens) * 0.5 // stronger penalty for extra unrelated tokens + } + + // Similarity scores to soften ordering among close matches + nameSim := normalizedSimilarity(nameLower, queryLower) + descSim := normalizedSimilarity(descLower, queryLower) + + var propSim float64 + for _, prop := range propertyNames { + if sim := normalizedSimilarity(prop, queryLower); sim > propSim { + propSim = sim + } + } + + searchText := nameLower + " " + descLower + if len(propertyNames) > 0 { + searchText += " " + strings.Join(propertyNames, " ") + } + fuzzySim := normalizedSimilarity(searchText, queryLower) + + score += nameSim * 2 + score += descSim * 0.8 + score += propSim * 0.6 + score += fuzzySim * 0.5 + + return score, matches.List() +} + +func getMaxResults(options []SearchOptions) int { + maxResults := DefaultMaxSearchResults + if len(options) > 0 && options[0].MaxResults > 0 { + maxResults = options[0].MaxResults + } + return maxResults +} + +func lowerInputPropertyNames(inputSchema any) []string { + if inputSchema == nil { + return nil + } + + // From the server, this is commonly a *jsonschema.Schema. + if schema, ok := inputSchema.(*jsonschema.Schema); ok { + if len(schema.Properties) == 0 { + return nil + } + out := make([]string, 0, len(schema.Properties)) + for prop := range schema.Properties { + out = append(out, strings.ToLower(prop)) + } + return out + } + + // From the client (or when unmarshaled), schemas arrive as map[string]any. + if schema, ok := inputSchema.(map[string]any); ok { + propsAny, ok := schema["properties"] + if !ok { + return nil + } + props, ok := propsAny.(map[string]any) + if !ok || len(props) == 0 { + return nil + } + out := make([]string, 0, len(props)) + for prop := range props { + out = append(out, strings.ToLower(prop)) + } + return out + } + + return nil +} + +type matchTracker struct { + list []string + seen map[string]struct{} +} + +func newMatchTracker(capacity int) *matchTracker { + return &matchTracker{ + list: make([]string, 0, capacity), + seen: make(map[string]struct{}, capacity), + } +} + +func (m *matchTracker) Add(part string) { + if _, ok := m.seen[part]; ok { + return + } + m.seen[part] = struct{}{} + m.list = append(m.list, part) +} + +func (m *matchTracker) List() []string { + return m.list +} + +func normalizedSimilarity(a, b string) float64 { + if len(a) == 0 || len(b) == 0 { + return 0 + } + + distance := fuzzy.LevenshteinDistance(a, b) + maxLen := len(a) + if len(b) > maxLen { + maxLen = len(b) + } + + similarity := 1 - (float64(distance) / float64(maxLen)) + if similarity < 0 { + return 0 + } + + return similarity +} + +func splitTokens(s string) []string { + if s == "" { + return nil + } + return strings.FieldsFunc(s, func(r rune) bool { + return r == '_' || r == '-' || r == ' ' + }) +} diff --git a/pkg/tooldiscovery/search_test.go b/pkg/tooldiscovery/search_test.go new file mode 100644 index 000000000..79d6fe8dd --- /dev/null +++ b/pkg/tooldiscovery/search_test.go @@ -0,0 +1,57 @@ +package tooldiscovery + +import ( + "testing" + + "github.com/google/jsonschema-go/jsonschema" + "github.com/modelcontextprotocol/go-sdk/mcp" + "github.com/stretchr/testify/require" +) + +func TestSearchTools_EmptyQueryReturnsNil(t *testing.T) { + results, err := SearchTools([]mcp.Tool{{Name: "issue_list"}}, " ") + require.NoError(t, err) + require.Nil(t, results) +} + +func TestSearchTools_FindsByName(t *testing.T) { + tools := []mcp.Tool{ + {Name: "issue_list", Description: "List issues"}, + {Name: "repo_get", Description: "Get repository"}, + } + + results, err := SearchTools(tools, "issue", SearchOptions{MaxResults: 10}) + require.NoError(t, err) + require.NotEmpty(t, results) + require.Equal(t, "issue_list", results[0].Tool.Name) +} + +func TestSearchTools_FindsByParameterName_JSONSchema(t *testing.T) { + tools := []mcp.Tool{ + { + Name: "unrelated_tool", + Description: "does something else", + InputSchema: &jsonschema.Schema{Properties: map[string]*jsonschema.Schema{"owner": {}}}, + }, + } + + results, err := SearchTools(tools, "owner", SearchOptions{MaxResults: 10}) + require.NoError(t, err) + require.NotEmpty(t, results) + require.Equal(t, "unrelated_tool", results[0].Tool.Name) +} + +func TestSearchTools_FindsByParameterName_MapSchema(t *testing.T) { + tools := []mcp.Tool{ + { + Name: "unrelated_tool", + Description: "does something else", + InputSchema: map[string]any{"properties": map[string]any{"repo": map[string]any{}}}, + }, + } + + results, err := SearchTools(tools, "repo", SearchOptions{MaxResults: 10}) + require.NoError(t, err) + require.NotEmpty(t, results) + require.Equal(t, "unrelated_tool", results[0].Tool.Name) +} diff --git a/third-party-licenses.darwin.md b/third-party-licenses.darwin.md index ca4c38f68..8217c7707 100644 --- a/third-party-licenses.darwin.md +++ b/third-party-licenses.darwin.md @@ -24,6 +24,7 @@ The following packages are included for the amd64, arm64 architectures. - [github.com/gorilla/css/scanner](https://pkg.go.dev/github.com/gorilla/css/scanner) ([BSD-3-Clause](https://github.com/gorilla/css/blob/v1.0.1/LICENSE)) - [github.com/josephburnett/jd/v2](https://pkg.go.dev/github.com/josephburnett/jd/v2) ([MIT](https://github.com/josephburnett/jd/blob/v1.9.2/LICENSE)) - [github.com/josharian/intern](https://pkg.go.dev/github.com/josharian/intern) ([MIT](https://github.com/josharian/intern/blob/v1.0.0/license.md)) + - [github.com/lithammer/fuzzysearch/fuzzy](https://pkg.go.dev/github.com/lithammer/fuzzysearch/fuzzy) ([MIT](https://github.com/lithammer/fuzzysearch/blob/v1.1.8/LICENSE)) - [github.com/mailru/easyjson](https://pkg.go.dev/github.com/mailru/easyjson) ([MIT](https://github.com/mailru/easyjson/blob/v0.7.7/LICENSE)) - [github.com/microcosm-cc/bluemonday](https://pkg.go.dev/github.com/microcosm-cc/bluemonday) ([BSD-3-Clause](https://github.com/microcosm-cc/bluemonday/blob/v1.0.27/LICENSE.md)) - [github.com/modelcontextprotocol/go-sdk](https://pkg.go.dev/github.com/modelcontextprotocol/go-sdk) ([MIT](https://github.com/modelcontextprotocol/go-sdk/blob/v1.2.0/LICENSE)) diff --git a/third-party-licenses.linux.md b/third-party-licenses.linux.md index 4cc59f2ed..981e388e5 100644 --- a/third-party-licenses.linux.md +++ b/third-party-licenses.linux.md @@ -24,6 +24,7 @@ The following packages are included for the 386, amd64, arm64 architectures. - [github.com/gorilla/css/scanner](https://pkg.go.dev/github.com/gorilla/css/scanner) ([BSD-3-Clause](https://github.com/gorilla/css/blob/v1.0.1/LICENSE)) - [github.com/josephburnett/jd/v2](https://pkg.go.dev/github.com/josephburnett/jd/v2) ([MIT](https://github.com/josephburnett/jd/blob/v1.9.2/LICENSE)) - [github.com/josharian/intern](https://pkg.go.dev/github.com/josharian/intern) ([MIT](https://github.com/josharian/intern/blob/v1.0.0/license.md)) + - [github.com/lithammer/fuzzysearch/fuzzy](https://pkg.go.dev/github.com/lithammer/fuzzysearch/fuzzy) ([MIT](https://github.com/lithammer/fuzzysearch/blob/v1.1.8/LICENSE)) - [github.com/mailru/easyjson](https://pkg.go.dev/github.com/mailru/easyjson) ([MIT](https://github.com/mailru/easyjson/blob/v0.7.7/LICENSE)) - [github.com/microcosm-cc/bluemonday](https://pkg.go.dev/github.com/microcosm-cc/bluemonday) ([BSD-3-Clause](https://github.com/microcosm-cc/bluemonday/blob/v1.0.27/LICENSE.md)) - [github.com/modelcontextprotocol/go-sdk](https://pkg.go.dev/github.com/modelcontextprotocol/go-sdk) ([MIT](https://github.com/modelcontextprotocol/go-sdk/blob/v1.2.0/LICENSE)) diff --git a/third-party-licenses.windows.md b/third-party-licenses.windows.md index c40c1dfcb..ae0e2389e 100644 --- a/third-party-licenses.windows.md +++ b/third-party-licenses.windows.md @@ -25,6 +25,7 @@ The following packages are included for the 386, amd64, arm64 architectures. - [github.com/inconshreveable/mousetrap](https://pkg.go.dev/github.com/inconshreveable/mousetrap) ([Apache-2.0](https://github.com/inconshreveable/mousetrap/blob/v1.1.0/LICENSE)) - [github.com/josephburnett/jd/v2](https://pkg.go.dev/github.com/josephburnett/jd/v2) ([MIT](https://github.com/josephburnett/jd/blob/v1.9.2/LICENSE)) - [github.com/josharian/intern](https://pkg.go.dev/github.com/josharian/intern) ([MIT](https://github.com/josharian/intern/blob/v1.0.0/license.md)) + - [github.com/lithammer/fuzzysearch/fuzzy](https://pkg.go.dev/github.com/lithammer/fuzzysearch/fuzzy) ([MIT](https://github.com/lithammer/fuzzysearch/blob/v1.1.8/LICENSE)) - [github.com/mailru/easyjson](https://pkg.go.dev/github.com/mailru/easyjson) ([MIT](https://github.com/mailru/easyjson/blob/v0.7.7/LICENSE)) - [github.com/microcosm-cc/bluemonday](https://pkg.go.dev/github.com/microcosm-cc/bluemonday) ([BSD-3-Clause](https://github.com/microcosm-cc/bluemonday/blob/v1.0.27/LICENSE.md)) - [github.com/modelcontextprotocol/go-sdk](https://pkg.go.dev/github.com/modelcontextprotocol/go-sdk) ([MIT](https://github.com/modelcontextprotocol/go-sdk/blob/v1.2.0/LICENSE)) diff --git a/third-party/github.com/lithammer/fuzzysearch/fuzzy/LICENSE b/third-party/github.com/lithammer/fuzzysearch/fuzzy/LICENSE new file mode 100644 index 000000000..dee3d1de2 --- /dev/null +++ b/third-party/github.com/lithammer/fuzzysearch/fuzzy/LICENSE @@ -0,0 +1,21 @@ +The MIT License (MIT) + +Copyright (c) 2018 Peter Lithammer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE.