Search Engine Optimization
Once your website is ready to be discoverable by search engines, you'll need to make sure your site is set up to be crawled and indexed. Making these changes does not guarantee that search engines will use your settings, but setting them up properly will influence search engines to show what you want them to show.
Set up your pages to be easily digested by search engines
In your page settings tab:
- Slug should be simple, descriptive, and hyphenated. See Keep a simple URL structure in Google's documentation for more detail
- Title tag (separate from page title) should also be simple and concise. Avoid keyword stuffing and repetition. This should be different on each page. See Influencing your title links in search results in Google's documentation for more detail and examples.
- Hide From Search Engines is unchecked by default, but if you happen to check this, it will prevent the page from being crawled/indexed.
In your page's Teasers tab:
- Teaser title should be a shorter title for listing blocks, and is optional.
- Meta description should be a short blurb describing your page. See Best practices for creating quality meta descriptions in Google's documentation for how to write quality meta description to get a higher chance of being used.
Below is an example of what Google designates as a quality snippet from Caltech's wiki page.
List your website on publicly available sites
Public sites that are already being crawled and indexed can help search engines to find your site. If your site is a personal or research website, make sure to get your site listed in your public profiles.
A great way to make your site public is to make sure your Caltech account is up-to-date in https://access.caltech.edu.
- Log in to https://access.caltech.edu.
- Under Self Service, click on My Personal Information.
- Under Name and Email tab, in the Email Address and Web Sites section, update Personal Website URL and/or Research Website URL.
- Click Submit.
Wait it a day or two, and your website URLs will be displayed in your directory profile on https://directory.caltech.edu. Sites that pull from directory will also gain access to your websites. For example, if your department already has a Caltech Site, and they have people pages that lists personal and research website URLs, your profile page should start showing your updated personal and/or research website URL when propagated.
Submit your site manually to Google search engine
Google being one of the most popular search engines, we will focus on how to get your site to show up when searching on Google. Keep in mind, Google does not promise when it'll be done, or whether they will use everything you provide. IMSS also cannot provide support and expertise beyond setting up your site to be crawled and indexed.
Ping your sitemap
The easiest way to let Google know about your website is to use their ping tool. Your site should automatically build a sitemap.xml for you. All you need to do is update the URL template below and enter it in the browser:
The FULL_URL_OF_SITEMAP can be found at
https://<sitename>.caltech.edu/sitemap.xml. For this website, the site map is located at https://sites.caltech.edu/sitemap.xml.
So to when we want to ping Google, we'll enter this into the browser:
For more options, see Submit your sitemap to Google.
Set Up Google Analytics and Google Search Console
If you are interested in tracking website traffic, our Preferences: Analytics Tab page will walk you through setting up Google Analytics 4. Once Google Analytics 4 is set up, you can connect it to Google Search Console, which will give you access to its URL Inspection Tool and allow you to:
- Check if a page is indexable.
- Troubleshoot why a page isn't being indexed.
- Request a specific page to be indexed.
Connecting to your Google Analytics 4 is super simple. Go to Google Search Console and click on Start now. You'll then enter your site URL and look up your existing Google Analytics 4 property. Click Continue to proceed.
You'll then be asked to verify ownership. Scroll down and expand Google Analytics and then verify. If you named your URL prefix the same as your Google Analytics property, it should find it and successfully link.
Property search bar allows you to select a verified property dashboard.
Once you're in the dashboard, you will finally have access to the URL inspection tool, as well as other useful indexing tools.
URL Inspection will take you to the top, where you can enter any page url on your site.
Page Index will give you a report of indexed pages and pages that couldn't be indexed. You should see the number of pages indexed gradually increase as you fix your pages.
Sitemaps will allow you to submit your sitemap. This is alternative to the ping tool mentioned above.
Removals will allow you to tell Google to take down a page or block search results. See Removals and SafeSearch reports Tool for details.
Once you have followed the instructions above, all you can do is wait for Google to pick it up. If you have Google Analytics, you should make sure traffic is coming through. If you have Google Search Console, then you can register your sitemap.xml and check when it was last read. Hopefully this will help you get your site crawled and index as quickly as possible.