Google search engine has a market share of over 60%. For some advanced features, such as searching metadata or relevant info of an object, we maybe want to integrate search result of Google search engine instead of inventing our own one. In this post, I would like to write down the steps how we can consume Google Custom Search API in .Net. The code itself is pretty short. However, because of lacking documentation it’s really time consuming to find out how the code should be, where to get the API Key or Search Engine ID for authentication. These all settings stuff drive me crazy because they locate on different control panel.
Please note that Custom Search Engine is a free API edition. For CSE users, the API provides 100 search queries per day for free. If you need more, you may sign up for billing in the Developers Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.
The other paid version is Google Site Search which is out of scope of this post.
1. Prerequisites
We need an Google Account. Sign up when you don’t have any. That’s all.
2. Get API key and write code
2.1 Get API key
If you already read my previous post Google Cloud Messaging, ASP.NET Web Api and Android client, you must be familiar with generating a Google API key. For Google Custom Search it’s almost the same.
– Go to https://console.developers.google.com
– Create a project if you don’t have any
– Click on your project link to go to project control panel. On the link menu, click on APIs, be sure that Custom Search API is enabled. If not you can enable it from available list under
– Also in project control panel, to Credentials, Create new key
– Be sure that a Browser key is generated
– Write down this browser key. We need it for code later.
2.2 Search Engine ID
The next thing we need for our code is a Search Engine ID. The API key above is used for authentication so that we’re allowed to use Google API function. For custom searching, we need to define our custom search engine to make a search.
– Go to https://www.google.com/cse/all and add new custom search engine
– For Sites to search enter any domain, for example google.com, then Create. In CSE control panel, select to edit the just created CSE. Be sure that you land on Setup menu for current editing CSE.
– Be sure that Search the entire web but emphasize included sites is selected then click on Update
– Click on Search Engine ID button to get its ID
– Write down this Search Engine ID, we’ll need it later.
2.3 Code
For consuming Google Custom Search API, luckily we don’t have to write our own code for reading/writing REST object. Google has also provided a Nuget package Google.Apis.Customsearch.v1 for .NET client. In your project, just simply add a reference to it over Nuget.
And the code is pretty simple.
private static void Main(string[] args)
{
const string apiKey = "AIzaSyAt8AkrmkiLVghrcKA3lFh37R79rSG0NsE";
const string searchEngineId = "003470263288780838160:ty47piyybua";
const string query = "hintdesk";
var customSearchService = new CustomsearchService(new BaseClientService.Initializer {ApiKey = apiKey});
var listRequest = customSearchService.Cse.List(query);
listRequest.Cx = searchEngineId;
Console.WriteLine("Start...");
IList<Result> paging = new List<Result>();
var count = 0;
while (paging != null)
{
Console.WriteLine($"Page {count}");
listRequest.Start = count * 10 + 1;
paging = listRequest.Execute().Items;
if (paging != null)
foreach (var item in paging)
Console.WriteLine("Title : " + item.Title + Environment.NewLine + "Link : " + item.Link +
Environment.NewLine + Environment.NewLine);
count++;
}
Console.WriteLine("Done.");
Console.ReadLine();
}
We have to initialize an instance of CustomsearchService with API key. Using this instance for generating a query of keyword hintdesk over Cse property. This query requires a Search Engine ID, so remember to set it’s Cx property by this value. At the end just simply call Execute() to get the search result
The image below shows a snapshot of keyword hintdesk search result.
The .NET client library supports also Async call. In combination with async, await keyword you can make an asynchronous call to Google web service.
3. Conclusions
Consuming Google API is pretty simple. The only annoyed thing is too little documentation to instruct when we can get die necessary settings such as API key or search engine id.
Source code: https://bitbucket.org/hintdesk/dotnet-how-to-use-google-custom-search-api
4. Updates
4.1 Paging
– Add paging to query to show more than top 10 result
Really nice, I couldn’t find how to do all of this. Really good manual!
Thanks
Really cool tutorial. thanks for that.
I need to access the right pane in the first result page when I search for location (eg. company name. G4S London, United Kingdom). I just need to pick the phone number and the website URL from there.
Any idea are appreciated.
Thank you,
It only displays english results and only first 10 results.
Thank You very much for your post…… It has reduced the complexity exponentially (n power n) to use the Google API …. Thank You ….
Thank you very much.
Documented so nicely, Really very easy to understand.
Thank you so much. I need search a image like this. do you know how can I do with this api
Thank you! What if I want to put it in a list? And foreach it in the view. How should I do it using MVC?
Thank you very mutch!!!
How do i make it display more than 10
@deaSTL: Code has been updated for paging.
Hi ! can you send me your code and how to make project step by step and make services and code and how to use API key’s etc
plzz Help me
Results from Google Custom Serach brower or Google Results are not matching with results from the code..Did anybody faced same issue?
@Rupali: Yes they will deliver different results because of hidden settings such as location, language… If you want to get the same result from Google, try to use this one
http://hintdesk.com/python-how-to-download-image-from-google-search/
nice one thanks
Thanks a lot for this tutorial. This saved me a lot of time.
Could you explain how it work paging?
Thank you very much for the contribution, I have a problem, when I do more than one search I always get the results of the previous search I always have to compile again to get clean results, how I can solve this issue?
Thanks in advance!
What does the Results object look like?
You can debug the code to see how the object looks like so that you can access the property you need. It’s a complicated class.
hello, thank you so much for your help. I’m just struggling to get access to more than title and link. how can I reach price, rating or price currency? can you please please please help me? 🙂
Really awesome tutorial, saved my time.
But there is an issue in paging, after fetching 100 records it is throwing error
listRequest.Start = count * 10 + 1; // when the value is 101 it will throw error
paging = listRequest.Execute().Items;
Error details :
Google.GoogleApiException
HResult=0x80131500
Message=Google.Apis.Requests.RequestError
Request contains an invalid argument. [400]
It would be great if you can help
But it give me only 100 search / day, how to get unlimited even by other means, kindly help.
I’m showing that Cse.List() won’t accept any parameters? Am I missing something?
Thanks for writing this, Tri!
You have this code twice:
paging != null
I’m not understanding how paging would possibly be null after the list is instantiated, and how the while loop does not create an endless loop, for where does paging ever become null?
What is the “Result” class of which the List is comprised?
To get the code to compile, I had to change this line:
CseResource.ListRequest listRequest = customSearchService.Cse.List(query); // no overload for “List” takes one argument
to:
CseResource.ListRequest listRequest = customSearchService.Cse.List();
I’ve had a couple of problems implementing the code; I don’t know if the api has changed since this was written, or it’s because my app is a .net Core one, or what. But I asked a question to the wider coding community at https://stackoverflow.com/questions/64654141/why-does-my-call-to-the-google-custom-search-api-fail-with-a-request-error-inva