Month: July 2018

How Visual Search Impacts the Future of SEO

How Visual Search Impacts the Future of SEO

Search in 2018 has become much more diverse than it was a few years ago, with new features such as voice search adding an entirely new dimension to how people explore search engines for answers to their queries. With AI Assistants like Google Assistant becoming more commonplace, search can now be conducted even without having to type a single letter.

Along with voice search, another new method on conducting a search is now being used as well in the form of visual search. If voice search is done through the human voice, visual search is done by simply opening your camera and taking a picture. This new way of conducting search through images might be a relatively new technology, but it has seen a good amount of development, with various apps and services testing the technology out. With this in mind, this is another new feature that aims to impact SEO in the future, and here’s what you need to know so far.

How Visual Search Works

Like voice search and standard search, visual search begins through input, and for this case, taking a picture and cropping it. While some people may compare visual search to Google’s image search feature and find some similarities, visual search offers much more. One of the most prominent apps that utilize visual search is Pinterest and Google, the latter of which offers Google Lens and Google Goggles. Here’s how some of them work:

Pinterest Visual Search

To start a visual search on Pinterest, tap on the camera icon on the search bar, and take a picture of what you want to have searched.

Pinterest Visual Search Camera

The photo will then be scanned for a couple of seconds to a minute, and then the results related to your image would pop up. Along with related images, you also get related tags, which allow you to view more options.

Pinterest Visual Search Results

Pinterest has one of the largest image libraries on the internet, and being able to tap into that database using visual search helps users find more useful images in the platform.

Google Googles

For Google Goggles, starting a visual search is similar, as the app is ready to capture images as soon as you turn it on. Along with taking photos, you can also pick images from your image gallery and have those scanned as well.

Google Googles Results

The process takes a couple of seconds depending on the image. Once scanning has been finished, you would be able to see a couple of results. Since the picture I took contains a logo of a popular brand, I am able to access search results related to the logo itself. Google Goggles also scans lettering on images as well, which allows you to translate various words and phrases into different languages.

While the technology is still relatively new and not many apps support it, visual search is something that works, and also something that users would use regularly in the near future the same way as voice search has become a regular fixture in a lot of places.

The Potential Impact of Visual Search

With the rise in popularity and use of voice search across the world, it would not be surprising to expect visual search to become widely popular as well. While voice search is useful in a way that it enables users to search without having to type anything, visual search helps users instantly identify various locations, items, and terms at a moment’s notice.

Back then, if a person wants to know more about a certain landmark or item, that person must search through a search engine. However, the problem would be that the person knows how the item or landmark looks like but does not know the name. This leads to various search queries that aim to describe what it looks like and leads to inconclusive results.

With visual search, these problems would now be solved, as taking a picture and running it through an app will instantly provide you with the best answers. While the technology still needs a few more improvements, visual search is readily available and can impact how we do search.

When it comes to businesses, visual search can play a huge role in driving sales and creating successful digital marketing campaigns. Imagine this scenario: A friend offered me a new drink that I immediately liked. Wanting to know more about that drink, I take a picture, and see the product itself, and where I could buy it. This could be done to a whole range of products as well, from clothing, furniture, toys, and even property.

E-commerce has become increasingly popular as of late, with more and more users enjoying the benefit of being able to shop items on the go. Adding visual search in the mix would help add another degree of interactivity and accessibility to the users and help drive traffic into websites.

Digital marketing campaigns will be able to benefit greatly from visual search, with brands being able to craft their campaigns around the use of visual search, which introduces people to new products or services in an interactive way. With features such as augmented reality also being utilized, the potential for these campaigns to go viral is also immense, as it would generate more buzz that leads to more conversions in the end.

Key Takeaway

Visual search may still be a very young technology, but it is definitely something to look out for, as it has the potential to positively impact the SEO landscape. With more apps and services making use of this handy feature, it would not be surprising to see it rise and become another integral part of search.

If you have questions and inquiries about Visual Search and SEO in general, leave a comment below and let’s talk.

How Visual Search Impacts the Future of SEO was originally posted by Video And Blog Marketing

Quality Scores for Queries: Structured Data, Synthetic Queries and Augmentation Queries

augmentation queries flowchart

Augmentation Queries

In general, the subject matter of this specification relates to identifying or generating augmentation queries, storing the augmentation queries, and identifying stored augmentation queries for use in augmenting user searches. An augmentation query can be a query that performs well in locating desirable documents identified in the search results. The performance of the query can be determined by user interactions. For example, if many users that enter the same query often select one or more of the search results relevant to the query, that query may be designated an augmentation query.

In addition to actual queries submitted by users, augmentation queries can also include synthetic queries that are machine generated. For example, an augmentation query can be identified by mining a corpus of documents and identifying search terms for which popular documents are relevant. These popular documents can, for example, include documents that are often selected when presented as search results. Yet another way of identifying an augmentation query is mining structured data, e.g., business telephone listings, and identifying queries that include terms of the structured data, e.g., business names.

These augmentation queries can be stored in an augmentation query data store. When a user submits a search query to a search engine, the terms of the submitted query can be evaluated and matched to terms of the stored augmentation queries to select one or more similar augmentation queries. The selected augmentation queries, in turn, can be used by the search engine to augment the search operation, thereby obtaining better search results. For example, search results obtained by a similar augmentation query can be presented to the user along with the search results obtained by the user query.

This past March, Google was granted a patent that involves giving quality scores to queries (the quote above is from that patent). The patent refers to high scoring queries as augmentation queries. Interesting to see that searcher selection is one way that might be used to determine the quality of queries. So, when someone searches. Google may compare the SERPs they receive from the original query to augmentation query results based upon previous searches using the same query terms or synthetic queries. This evaluation against augmentation queries is based upon which search results have received more clicks in the past. Google may decide to add results from an augmentation query to the results for the query searched for to improve the overall search results.

How does Google find augmentation queries? One place for it to look is at query logs and click logs. As the patent tells us:

To obtain augmentation queries, the augmentation query subsystem can examine performance data indicative of user interactions to identify queries that perform well in locating desirable search results. For example, augmentation queries can be identified by mining query logs and click logs. Using the query logs, for example, the augmentation query subsystem can identify common user queries. The click logs can be used to identify which user queries perform best, as indicated by the number of clicks associated with each query. The augmentation query subsystem stores the augmentation queries mined from the query logs and/or the click logs in the augmentation query store.

This doesn’t mean that Google is using clicks to directly determine rankings But it is deciding which augmentation queries might be worth using to provide SERPs that people may be satisfied with.

There are other things that Google may look at to decide which augmentation queries to use in a set of search results. The patent points out some other factors that may be helpful:

In some implementations, a synonym score, an edit distance score, and/or a transformation cost score can be applied to each candidate augmentation query. Similarity scores can also be determined based on the similarity of search results of the candidate augmentation queries to the search query. In other implementations, the synonym scores, edit distance scores, and other types of similarity scores can be applied on a term by term basis for terms in search queries that are being compared. These scores can then be used to compute an overall similarity score between two queries. For example, the scores can be averaged; the scores can be added; or the scores can be weighted according to the word structure (nouns weighted more than adjectives, for example) and averaged. The candidate augmentation queries can then be ranked based upon relative similarity scores.

I’ve seen white papers from Google before mentioning synthetic queries, which are queries performed by the search engine instead of human searchers. It makes sense for Google to be exploring query spaces in a manner like this, to see what results are like, and using information such as structured data as a source of those synthetic queries. I’ve written about synthetic queries before at least a couple of times, and in the post Does Google Search Google? How Google May Create and Use Synthetic Queries.

Implicit Signals of Query Quality

It is an interesting patent in that it talks about things such as long clicks and short clicks, and ranking web pages on the basis of such things. The patent refers to such things as “implicit Signals of query quality.” More about that in the patent here:

In some implementations, implicit signals of query quality are used to determine if a query can be used as an augmentation query. An implicit signal is a signal based on user actions in response to the query. Example implicit signals can include click-through rates (CTR) related to different user queries, long click metrics, and/or click-through reversions, as recorded within the click logs. A click-through for a query can occur, for example, when a user of a user device, selects or “clicks” on a search result returned by a search engine. The CTR is obtained by dividing the number of users that clicked on a search result by the number of times the query was submitted. For example, if a query is input 100 times, and 80 persons click on a search result, then the CTR for that query is 80%.

A long click occurs when a user, after clicking on a search result, dwells on the landing page (i.e., the document to which the search result links) of the search result or clicks on additional links that are present on the landing page. A long click can be interpreted as a signal that the query identified information that the user deemed to be interesting, as the user either spent a certain amount of time on the landing page or found additional items of interest on the landing page.

A click-through reversion (also known as a “short click”) occurs when a user, after clicking on a search result and being provided the referenced document, quickly returns to the search results page from the referenced document. A click-through reversion can be interpreted as a signal that the query did not identify information that the user deemed to be interesting, as the user quickly returned to the search results page.

These example implicit signals can be aggregated for each query, such as by collecting statistics for multiple instances of use of the query in search operations, and can further be used to compute an overall performance score. For example, a query having a high CTR, many long clicks, and few click-through reversions would likely have a high-performance score; conversely, a query having a low CTR, few long clicks, and many click-through reversions would likely have a low-performance score.

The reasons for the process behind the patent are explained in the description section of the patent where we are told:

Often users provide queries that cause a search engine to return results that are not of interest to the users or do not fully satisfy the users’ need for information. Search engines may provide such results for a number of reasons, such as the query including terms having term weights that do not reflect the users’ interest (e.g., in the case when a word in a query that is deemed most important by the users is attributed less weight by the search engine than other words in the query); the queries being a poor expression of the information needed; or the queries including misspelled words or unconventional terminology.

A quality signal for a query term can be defined in this way:

the quality signal being indicative of the performance of the first query in identifying information of interest to users for one or more instances of a first search operation in a search engine; determining whether the quality signal indicates that the first query exceeds a performance threshold; and storing the first query in an augmentation query data store if the quality signal indicates that the first query exceeds the performance threshold.

The patent can be found at:

Query augmentation
Inventors: Anand Shukla, Mark Pearson, Krishna Bharat and Stefan Buettcher
Assignee: Google LLC
US Patent: 9,916,366
Granted: March 13, 2018
Filed: July 28, 2015

Abstract

Methods, systems, and apparatus, including computer program products, for generating or using augmentation queries. In one aspect, a first query stored in a query log is identified and a quality signal related to the performance of the first query is compared to a performance threshold. The first query is stored in an augmentation query data store if the quality signal indicates that the first query exceeds a performance threshold.

References Cited about Augmentation Queries

These were a number of references cited by the applicants of the patent, which looked interesting, so I looked them up to see if I could find them to read them and share them here.

  1. Boyan, J. et al., A Machine Learning Architecture for Optimizing Web Search Engines,” School of Computer Science, Carnegie Mellon University, May 10, 1996, pp. 1-8. cited by applicant.
  2. Brin, S. et al., “The Anatomy of a Large-Scale Hypertextual Web Search Engine“, Computer Science Department, 1998. cited by applicant.
  3. Sahami, M. et al., T. D. 2006. A web-based kernel function for measuring the similarity of short text snippets. In Proceedings of the 15th International Conference on World Wide Web (Edinburgh, Scotland, May 23-26, 2006). WWW ’06. ACM Press, New York, NY, pp. 377-386. cited by applicant.
  4. Ricardo A. Baeza-Yates et al., The Intention Behind Web Queries. SPIRE, 2006, pp. 98-109, 2006. cited by applicant.
  5. Smith et al. Leveraging the structure of the Semantic Web to enhance information retrieval for proteomics” vol. 23, Oct. 7, 2007, 7 pages. cited by applicant.
  6. Robertson, S.E. Documentation Note on Term Selection for Query Expansion J. of Documentation, 46(4): Dec. 1990, pp. 359-364. cited by applicant.
  7. Talel Abdessalem, Bogdan Cautis, and Nora Derouiche. 2010. ObjectRunner: lightweight, targeted extraction and querying of structured web data. Proc. VLDB Endow. 3, 1-2 (Sep. 2010). cited by applicant .
  8. Jane Yung-jen Hsu and Wen-tau Yih. 1997. Template-based information mining from HTML documents. In Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative application of artificial intelligence (AAAI’97/IAAI’97). AAAI Press, pp. 256-262. cited by applicant .
  9. Ganesh, Agarwal, Govind Kabra, and Kevin Chen-Chuan Chang. 2010. Towards rich query interpretation: walking back and forth for mining query templates. In Proceedings of the 19th international conference on World wide web (WWW ’10). ACM, New York, NY USA, 1-10. DOI=10. 1145/1772690. 1772692 http://doi.acm.org/10.1145/1772690.1772692. cited by applicant.

This is a Second Look at Augmentation Queries

This is a continuation patent, which means that it was granted before, with the same description, and it now has new claims. When that happens, it can be worth looking at the old claims and the new claims to see how they have changed. I like that the new version seems to focus more strongly upon structured data. It tells us that it might use structured data in sites that appear for queries as synthetic queries, and if those meet the performance threshold, they may be added to the search results that appear for the original queries. The claims do seem to focus a little more on structured data as synthetic queries, but it doesn’t really change the claims that much. They haven’t changed enough to publish them side by side and compare them.

What Google Has Said about Structured Data and Rankings

Google spokespeople had been telling us that Structured Data doesn’t impact rankings directly, but what they have been saying does seem to have changed somewhat recently. In the Search Engine Roundtable post, Google: Structured Data Doesn’t Give You A Ranking Boost But Can Help Rankings we are told that just having structured data on a site doesn’t automatically boost the rankings of a page, but if the structured data for a page is used as a synthetic query, and it meets the performance threshold as an augmentation query, it might be shown in rankings, thus helping in rankings (as this patent tells us.)

Note that this isn’t new, and the continuation patent’s claims don’t appear to have changed that much so that structured data is still being used as synthetic queries, and is checked to see if they work as augmented queries. This does seem to be a really good reason to make sure you are using the appropriate structured data for your pages.


Copyright © 2018 SEO by the Sea ⚓. This Feed is for personal non-commercial use only. If you are not reading this material in your news aggregator, the site you are looking at may be guilty of copyright infringement. Please contact SEO by the Sea, so we can take appropriate action immediately.
Plugin by Taragana

Quality Scores for Queries: Structured Data, Synthetic Queries and Augmentation Queries was originally posted by Video And Blog Marketing

How we did an emergency HTTPS migration using the ODN to avoid Chrome security warnings [case study]

Getting changes made in enterprise environments is hard, even when there are clear financial impacts of not making the changes. Anyone who hasn’t migrated to HTTPS by this point, is aware of the need; it hasn’t happened yet because of insurmountable blockers like mixed-content warnings in hard-to-update back-end systems.

If this sounds like you, read on because the architecture of the ODN, deploying as a CDN, or between your CDN and origin, means that it’s agnostic to whatever server-side technologies you are using, and whatever CMS you have in place, so no matter what limitations your tech stack is imposing, the ODN can help get past these kinds of blockers and allow you to migrate quickly to HTTPS if you haven’t already done so. Get in touch if you want to learn more or see a demo of the ODN.

CONTACT US TO LEARN MORE ABOUT THE ODN

With the rollout of Chrome 68 highlighting all HTTP sites as not secure, there has been widespread press about some sites getting “flagged” (here is the BBC highlighting the Daily Mail in their headline and calling out half a dozen retailers by name).

Sometimes companies behave just like the people that make them up. Most of us can remember a time when we’ve left that big piece of work until really close to the deadline, or even ended up starting work once it’s arguably a tiny bit too late. And businesses do the same – whether it’s shipping the GDPR-related privacy policy update on May 24th (yeah, ok, we did that), or fixing mobile-friendliness issues in a frantic mobilegeddon-related rush, what’s important is too often left until it becomes urgent.

In the case of HTTPS migrations, though, there are a range of reasons why it can actually be really hard to get them done in an enterprise environment. It’s common to have an organisational desire to get this done, but to have specific technical blockers. So, with the growing urgency coming from the external changes, we’ve been looking for ways to live up to our core values and effect change and get things done. In alignment with this, we recently got an urgent HTTPS migration done for a major retailer by using our ODN platform to mitigate a range of technical and on-page blockers. Here’s how:

On-page changes

One of the most common blockers to an HTTPS migration in enterprise environments is fixing mixed-content warnings where your newly-HTTPS pages rely on assets or scripts that are still loaded over HTTP. Even once you have your images (for example) also moved over to a secure hosting environment, you still need to update all the references to those images to use their HTTPS URLs.

We have used the ODN to:

  • Update image links from HTTP to HTTPS
  • Modify the embed codes and script references for 3rd party plugins
  • Update inline CSS references to HTTP assets

By being able to do this site-wide, across all pages sharing a particular template, or on specific pages, we get the right blend of power and efficiency that enables a large volume of mixed-content warnings to be resolved in a short period of time.

Fixing meta information

There’s a variety of meta information that might need to be updated during the migration to HTTPS, but probably the most important is the canonical and hreflang information. The ODN can inject this information into pages where it’s missing (including into the headers for PDFs, for example), and update existing links to the new scheme.

Since canonical and hreflang links are poorly-handled by many CMSs, the power of being able to fix this “outside the system” is powerful and can be set up as a final check to ensure correct canonical links.

Setting up redirects

A critical part of the deployment of a migration to HTTPS is the 1-1 page-level redirects from HTTP pages to HTTPS pages. It’s common for this to be hard to manage, because you may well want to prevent your origin server from even responding to port 80 (HTTP) requests in the new secure world, which means your server can’t handle the redirects needed. We can serve them for you, and make sure that every request hitting your origin is port 443 (HTTPS).

It’s possible to set up redirect rules at the edge with a CDN, but our platform brings two main benefits over that approach:

  1. if you are migrating sections of your site at a time, we and flexibly update the rules for complex groups of pages
  2. we can add logic to avoid chained redirects which is often difficult with blanket rules.

Adding and modifying headers

Content Security Policy (CSP) headers are an important part of many HTTPS setups, and in particular, in risk-averse environments, you may well want to use a changing set of CSP headers to roll out HTTPS cautiously:

  • Roll out initially with a very lax CSP that allows insecure assets, but reports them via the report-uri policy directive
    • This means, that on any HTTPS page that uses HTTP resources, the browser will still report the page as insecure but it will work and you will get collect data on which resources are still in use where
  • As you then remove all HTTP dependencies, you can tighten up the CSP to much stricter policies and achieve the “secure” label in the browser
    • You may modify this on a section-by-section basis as each section meets the technical requirements
  • Once all pages are fully on HTTPS and redirects are in place, you can add HSTS (Strict-Transport-Security) to the mix
    • HSTS is a header served on the HTTPS version of your site that is cached by browsers and informs them not to trust the HTTP version in future and always to request the HTTPS version of every page on your site (until the expiry of the HSTS setting)

It can be difficult in many hosting environments to achieve this level of granularity, control, and agility with changes to headers, and the ODN can help with controlling them at the page, template, or domain level.

Want to see it first-hand?

The architecture of the ODN, deploying as a CDN, or between your CDN and origin, means that it’s agnostic to whatever server-side technologies you are using, and whatever CMS you have in place, so no matter what limitations your tech stack is imposing, the ODN can help fix up these kinds of blockers.

If you are in an environment where you are blocked from getting important things done by a lack of agility for on-page and server configuration changes, we might be able to help. Drop us a line if you would like to see our ODN platform in action.

CONTACT US TO LEARN MORE ABOUT THE ODN

How we did an emergency HTTPS migration using the ODN to avoid Chrome security warnings [case study] was originally posted by Video And Blog Marketing

Getting an HTTPS migration done in an enterprise environment

There have been some excellent articles written about the steps necessary for a successful HTTP to HTTPS migration. Although we know that a move is becoming more and more pressing, knowing what to do is only a small part of the story when you’re working in an enterprise environment. Somehow, we need to figure out at the very least:

  • Who do we need to persuade, and what’s going to convince them?
  • How are we going to mitigate risk as much as possible?
  • How are we going to get the actual details done? – some of these steps are simple but hard

Many of you will be living this, and be feeling these challenges keenly. We’ve been putting a lot of thought and a lot of work into helping our points of contact make these cases and get these migrations done. Here are some pointers and tips we’ve learned along the way; hopefully they’ll help you.

Sidenote: read about how we used our ODN platform to help to ship an urgent HTTPS migration for a major retailer

If you want to get some sense of the challenges, or if you don’t regularly work with large sites in complex organisations, the journey of the BBC to secure their news section might give you some idea of the complexity:

  1. Two years ago, they talk about making changes at their CDNs to enable HTTPS in the future when the individual products (e.g. homepage or travel news) get to the point of being ready on the back-end
  2. By the end of 2017, they are talking about enabling HTTPS to their origins and worrying about how to warm up the HTTPS caches
  3. June 2018 we get the Medium post about the elusive padlock on BBC News after problems like an Indian government-mandated network block that rendered the site totally inaccessible

And then even after all that effort, we realise that the first links I shared there are on the “BBC blogs” section of the site which is still insecure:

Making the case for the enterprise HTTPS migration

In some cases, I find that business cases and return on investment are the most powerful drivers of change, and there are possible approaches that could use data to make this case (looking first at drops in conversion rate from warnings over unsecured pages) but my first approach would be an argument that looks more like this:

  1. We’re definitely going to have to do this eventually
  2. External changes mean that we shouldn’t keep putting it off – there are reputational, business, and operational risks from delaying

It’s a more risk-averse argument focusing on avoidance of downside, but it has powerful emotional elements to it:

1. We’re definitely going to have to do this eventually

There are plenty of rational arguments for the move to HTTPS (great article) but this is mainly an argument that no matter what decisions we make, we can’t put this off forever. We can look at competitors, large sites, and external moves (e.g. by Chrome) to make this point powerfully:

Websites are moving to HTTPS at unprecedented rates

Google research shows that:

  • More than half of large sites now have HTTPS available (moving from 39% to 54% in the year to Feb 2017) with default HTTPS doubling in a single year
  • The bigger / more popular a site is, the greater its chance of having HTTPS available and the greater the chance of it using HTTPS by default
  • A majority of desktop browsing now occurs over HTTPS

All of which means that users are becoming more accustomed to seeing HTTPS everywhere and increasingly expect it. We have even seen this in qualitative feedback from website user testing (create a free account to watch this video):

High ranking websites are particularly likely to be HTTPS

In just 9 months, after announcing HTTPS as a (minor) ranking factor, the % of HTTPS results on page 1 of Google search results jumped from 30% to over 50%:

New features increasingly assume HTTPS connections

Features like HTTP/2 (which can bring significant speed improvements to many sites), and service workers (which are required for app-like capabilities such as offline functionality) require or assume the presence of HTTPS connections. If you aren’t already up to speed on them, this presentation by our VP Product, Tom Anthony will tell you what you need to know (create a free account to watch this video).

2. External changes mean we should do it now

Browser changes increase the urgency of making the change

We have known for some time that Google in particular was going to use their Chrome browser to push webmasters to HTTPS. Initially, the just flagged sites as insecure if they were on HTTP when a form was detected:

They then announced further changes to take it from just those pages to any HTTP page:

This actually isn’t Google’s last planned update on this theme, there will be a release of Chrome sometime soon that highlights the insecurity in red:

Not only is this change raising the profile of your security setup with your users and customers and most likely hurting engagement and conversion rate, but it is starting to bring bad press down on those who haven’t made the move yet. This BBC article, for example calls out a number of sites by name and cautions that while you shouldn’t necessarily entirely avoid sites that are still on HTTP, “you should be wary on those that require you to sign in or which let you buy goods and services through them”.

Mitigating the risk of an HTTPS migration

OK, so we know it’s something we want to do, and key stakeholders are coming around to the idea, but pretty early in the process, someone is going to bring up risk factors, and how we can minimise and mitigate as many of the risks as possible.

Aside from thorough testing in a staging environment, what else can you do to reduce the risks of going to HTTPS? One key tool in the arsenal is Content Security Policy (CSP) headers. One of the hardest parts of the move is avoiding mixed-content warnings, where your (secure) page references HTTP resources and assets. A good way of mitigating risks and avoiding UI issues and broken functionality from blocked assets is to roll out HTTPS initially with a very lax CSP that allows insecure assets, but reports them via the report-uri policy directive. This means, that on any HTTPS page that uses HTTP resources, the browser will still report the page as insecure but it will work and you will get collect data on which resources are still in use where.

As you then remove all HTTP dependencies, you can tighten up the CSP to much stricter policies and achieve the “secure” label in the browser. Once all pages are fully on HTTPS and redirects are in place, you can add HSTS (Strict-Transport-Security) to the mix. HSTS is a header served on the HTTPS version of your site that is cached by browsers and informs them not to trust the HTTP version in future and always to request the HTTPS version of every page on your site (until the expiry of the HSTS setting).

(Note: the more important security is to your site, the further down this rabbit-hole you may wish to go – right up to preloaded HSTS sites – though note that this is not easily reversible even temporarily in the event of certificate errors.)

There are a variety of great resources on the SEO details, with checklists and processes to follow, so I’m not going to repeat all of the steps here. I recommend:

  • This article on all the benefits of HTTPS and technical features you can use once you have moved over
  • Patrick Stox outlined the process, and Aleyda published a great checklist on the SEO steps and implications
  • You may find it useful to refer to the official Google line (from John Mueller) to reassure stakeholders about Google’s view of the process and its benefits
    • THOUGH I am very concerned about the advice to “use 302 redirects + rel=canonical to HTTP if you want to test HTTPS but not have it indexed”. I would not recommend ever having canonical links that point to pages that redirect back to the original page (even 302 redirects). I would recommend not doing this.

How are we going to get the details implemented?

As always, knowing what to do and getting agreement to go ahead is only a small part of the battle in many organisations. Large websites and big companies typically have myriad dependencies and integrations of older systems that throw up unexpected roadblocks in the way of the objective. In the case of an HTTPS migration, this is often things like:

  • We have mixed-content warnings that we can’t deal with at scale – how are we going to update all the references to images on http URLs? What about our 3rd-party plugins and embeds?
  • Our canonical links all point to the HTTP version of our site and the engineering work to update them across all the different page templates is going to add scary amounts of money to the cost of this project – potentially across multiple back-ends / CMS
  • We want to add Referrer-policy and especially Content Security Policy headers to enable better testing and mitigate risks, but we have no way to control HTTP headers through our CMS

Recommendations of what to do are worthless if you can’t get them done – a mantra that we repeat a lot at Distilled is it’s not our job to deliver reports – it’s our job to effect change. One of the ways we’ve done this is by building the ODN platform which makes it easy to make agile changes to HTML and HTTP responses. We just completed an urgent HTTPS migration for a major retailer where we addressed exactly these kind of blockers with the platform – you can read more about that here.

If you’re in the unfortunate situation of knowing you need to make the move to HTTPS, and having the organisation aligned, but being blocked by these kinds of technical issue, drop us a line to discuss whether we can help.

CONTACT US TO LEARN MORE ABOUT THE ODN

Getting an HTTPS migration done in an enterprise environment was originally posted by Video And Blog Marketing

Going beyond Marketing Segments: Q&A with Stanford’s David L. Jaffe

Business Spotlight

The struggle to rank is real.

 

Performing well in search engine results through search engine optimization requires a massive amount of research and planning – and even more hard work. It requires a person to scour over analytic data, to identify the right marketing segments, and to create an effective plan to reach and satisfy target audiences.

And inside all the marketing verbiage, it’s easy to lose track of the original goal. Sometimes you need a new perspective.

We had the pleasure of speaking with David L. Jaffe, a lecturer in Mechanical Engineering at Stanford University about his course “Perspectives in Assistive Technology,” a class that designs solutions to benefit people with disabilities and older adults. His thoughts are especially poignant when it comes to marketing; to succeed, identifying a marketing segment for your product isn’t enough.

Search engines try to organize information and help their users. If you’re not sincerely attempting to help those very same people, you might find the process of ranking exceedingly difficult.

About David L. Jaffe

David L. Jaffe holds a BS degree in Electrical Engineering from the University of Michigan and a MS degree in Biomedical Engineering from Northwestern University.

Dave was a Research Biomedical Engineer at the VA Palo Alto Health Care System’s Rehabilitation Research and Development Center. There his interests were designing, developing, testing, and bringing to market microcomputer-based devices for veterans with disabilities including communication, mobility, and information systems.

At Stanford, he teaches “Perspectives in Assistive Technology” and coaches students in other courses who are working on team projects related to assistive technology. He can be reached via email at davejaffe@stanford.edu.

Could you describe your Stanford course, Perspectives in Assistive Technology, to those who aren’t familiar with it?

Perspectives in Assistive Technology (ENGR110/210) is a 10-week course that explores the design, development, and use of technology that benefits people with disabilities and older adults.

In this course, Stanford students from many disciplines and all years learn about disability, assistive technology, and rehabilitation engineering from guest lecturers including healthcare professionals, researchers, and people with disabilities (including students) and older adults. Other class sessions feature field trips to local facilities, a movie night, and an assistive technology fair.

Enrolled students have the opportunity to work on projects that benefit people with a disability, older adults, and their caregivers in the local community. These projects typically address difficulties in performing tasks such as working, learning, moving, communicating, accessing products (including computers), and other daily living activities including cooking, cleaning, creative expression, and pursuing happiness. Projects that explore design concepts that improve diagnosis, therapy, and rehabilitation are also pursued.

Projects typically address one individual’s challenges through the fabrication and testing of a functional prototype. However due to the short course duration, commercialization is not pursued.

More information about the course can be found at: http://engr110.stanford.edu

Do engineers have a unique method for problem-solving?

Each engineering problem-solving situation is somewhat different, depending on the problem itself: its purpose (student project, research question, product manufacturing), goal(s), and time & money budgeted.

However, in general it is important to fully understand the problem by bringing together skilled team members and an individual those who would benefit from the solution. A review of current technology and existing products is critical.

Next steps would include developing design criteria, identifying current technology which could be employed, and brainstorming possible solutions.

The design development phase consists of the fabrication, testing, analysis, and redesign of increasingly refined prototypes until the design goals are met (or time or money runs out).

How do you use that engineering design process in your Assistive Technology course?

The process employed in the course includes the above with emphasis on working closely with the project partner, a person with a disability or older adult in the local community. Each student team is expected to meet with the instructor to make sure they are making progress toward the project goal and to provide suggestions.

Your class exposes students to new people, ideas, and problems. How important do you think that is to the process of considering and pursuing solutions to problems and challenges that people with disabilities and older adults experience?

It is supremely important to present student teams with real challenges affecting real people in the community. Much of students’ traditional class experiences are limited to solving problem sets, examining case study descriptions, and considering made-up situations involving imaginary people. Community-based project courses like Perspectives in Assistive Technology give students a real-world experience to prepare them for their professional careers.

What are the skills that you’d most like to instill in your students?

An excellent question!

Several years ago, I realized there was a huge disconnect between what I learned as a student in the universities I attended and what I do now as a professional. In particular, no one told me how important it is for a prospective engineer to master writing and presentation skills. So, I decided to promote professional skills to students in my course. I use assistive technology as a framework for introducing, exercising, and improving these skills: working in a team, working in the community, problem solving, following an engineering design process, employing critical thinking, and practicing report writing & project presentations communication skills.

I have done away with problem sets, quizzes, readings from text books, and exams – skills that students have already mastered and are much less important on the job.

There’s a clear need for more recognition of and services supporting older adults and people who live with a disability. What are some common misconceptions that the public have of these groups?

One common misconception is considering people with disabilities and older adults as “groups” or “markets”. In reality, they are individuals, all different in their situations, abilities, life expectations, desires, and preferences.

Do we do enough as a society to include them?

I don’t think so. For example, I find it amazing that people with disabilities and older adults are rarely included in discussions of diversity, despite their representation nationally (19% and 15% respectively). It is important to realize that everyone has something to contribute – perhaps as a valued employee, a loving family member, an active community participant, or as a consumer.

What types of solutions do they need the most?

As I mentioned, people with disabilities and older adults are individuals – each with their own challenges. Some challenges might be lessened by an appropriate assistive technology device, others by financial, medical, social services, or educational support.

I noticed in your class notes that you provide resources to improve critical thinking for your students. How important is critical thinking to the process of addressing a problem or challenge experienced by a person with a disability or older adult?

Critical thinking, especially in engineering, can inform the design of good assistive technology solutions.

How important are they to Assistive Technology?

Critical thinking, good teamworking skills, as well as familiarity with the engineering design process are all key to creating suitable assistive technology solutions and useful products.

Going beyond Marketing Segments: Q&A with Stanford’s David L. Jaffe was originally posted by Video And Blog Marketing

What do dolphins eat? Lessons from how kids search

I recently came across a couple of fascinating papers (here and here) all about how kids search. I found it fascinating in its own right, and also found it thought-provoking in the new ways of searching it showed that had simply never occurred to me. Here are some of the most interesting things I found (though it’s remarkably accessible, and you should totally read the whole thing).

The researchers studied children aged 7-11, and of varying degrees of experience and comfort with the web and with computer-based research. In the course of their study, they identified seven “search roles” (almost like personas) that children display when seeking information:

Many of these are fairly self-explanatory on the surface (though it’s always interesting to read the details) and you may even identify with some of them yourself, as an adult. One of the most interesting to me was what they called the visual searcher.

People don’t all think like you

This was a mode of search that I had rarely found myself in, and had barely even considered could be a thing outside of certain forms of specific image search (e.g. [microsoft logo]). What they found was a cohort of children who turned first to image search for a wide range of their information-gathering needs. In some cases, this appeared to be motivated by discomfort with text and with reading, or at least with scanning and reading fast. In others, though, it seemed to be about veracity and trusting only what you have seen with your own eyes. For those of us who know people who write on the internet, maybe this isn’t the craziest instinct.

One example that has stayed in my mind since I read about it is the experience of certain kids when asked to answer the question what do dolphins eat?

The anecdote that stood out for me was the child who not only turned to image search to answer the question, but did the one-word image search [dolphin] and then scrolled down through pages of results until, having found a picture of a dolphin eating something, turned to the researcher to declare triumphantly that dolphins eat fish.

The lesson here is clearly about the power of observing real-world users. This is the kind of insight that is hard to glean from the raw data of keyword research. Even if you figure out that there is image search volume for [dolphin], you’re some way from the insight that someone is searching for information about what they eat.

This era (the research was published in 2010) was marked by a wide range of qualitative research coming out of Google. I might dive deeper into some other research in another post, but for now, onto the next insight.

There are searches that are hard, and people are failing to complete them

In my presentation and post the next trillion searches, I talked about the incremental search volume available in the coming years as technology progresses to the point that it can satisfy intents, and answer questions that current technology cannot:

One of the things I didn’t talk about in that post was the times that current searcher intent is not fulfilled even though the information is out there and today’s technology is more than capable of finding it. To understand more about what I mean here, let’s take another look at search challenges for kids:

For a start, it’s worth noting that Google can’t answer this query outright. Unlike with more and more factual queries, Google is not able to return a one-box with any answer, never mind the correct answer.

Unsurprisingly, kids struggled with this one (as I suspect would many adults). It tests their ability to string together a sequence of queries, each one building on the last, to discover the answer at the end of the rainbow. And along the way, they have to be sceptical of the information they come across and not get distracted by the pots of fools’ gold:

At certain points along the way, our intrepid searcher may come across pages that purport to give the answer, but which in fact do not for a variety of reasons (not least, as with the example above, that this information can fall easily out of date).

So it combines the ability to break down a question into structured thoughts, achieve complex stringing together of queries, and avoid pitfalls of incorrect and misleading information along the way. How many adults do you know who might trip up on this?

Amazingly, some of the older kids in the study managed to find the correct answer.

If you have kids in your life, try this out

If you have kids, or you have younger siblings, cousins, nieces, nephews, etc. I’d strongly encourage anyone interested in search to sit and watch them take on relatively undirected searching tasks while you watch. I think it’s pretty educational (for them!), but I also think there’s a good chance you will learn a good deal. In particular, since this research was done in 2010, it appears to have been entirely desktop-driven. I’d be interested in the mobile-first version if anyone wants to run it and write it up!

Anyway, it turns out my kids are (roughly) in the right age range – at the time of experimenting, my daughter was just turned 8, and my son was 5. My daughter was therefore in the age range, and it was interesting to see how she fared:

Rachel aged 8

She found it fairly easy to find out what dolphins eat. Google coped fine with her misspelling of “dolfin” and she wasn’t fazed by the results coming back for the correct spelling. She didn’t bother reading the “showing results for…” section (nor the paid ad, natch) and skipped straight to the one-box. She scanned it without reading aloud and then answered the question: telling me some things dolphins eat. In the process she went from an unmotivated searcher to a motivated searcher: she got intrigued by what a cephalopod is (it is mentioned in the one-box) and set of on an unprompted search to find out.

The next task was too much for her. She’s British, so I decided to go with prime minister, as I didn’t think she’d know what or who the vice president was. It turns out she wasn’t entirely clear on what a prime minister is either, searching for primeinister. She composed a search that could have worked as a stand-alone query: Google corrected it to [when is the prime minister’s birthday next year]. In fact, Google couldn’t answer this directly, and since it wasn’t quite the actual answer to the question as asked, she got stuck at this point, unable to structure the query quite how she wanted it.

Actually, she probably went slightly too far in the first jump. She probably should have gone with something like [when is the prime minister’s birthday] and followed with [what day is <date> next year] but she didn’t make that logical leap unprompted.

Even though my son was a little young, we thought it’d be fun to see how he fared on the “dolphin” question. The date one was a little too much of a stretch:

Adam aged 5

Interestingly, he spelled “dolfin” the same way as his sister (this must be our failing as parents!) but also went with the phonetic “wat” instead of “what”. Nonetheless, Google was quite happy interpreting his search as [what do dolphins eat] so he got the same one-box as his sister.

Just like her, he skipped everything else on the page to go straight to the one-box. This is probably not that surprising in either of their cases – it’s most likely what adults do, and it’s clearly designed to draw attention with the bright image high up on the page.

What was interesting and different was that he didn’t read the whole thing. At the time of the experiment, he was obviously a less confident reader, and preferred to read aloud rather than in his head. He didn’t scan the one-box for the answer and report it, but interestingly, nor did he read the one-box aloud. Instead, he read only the words in bold.

This isn’t the most obviously crazy strategy (at least in the mind of a 5 year old): it isn’t crazy to think that Google would have bolded the words that are the answers to the question you asked, though search professionals know that’s not what’s really going on here. It started okay but then went a little bit off the rails. Here’s what he read out as the answer to [what do dolphins eat?]:

  • Fishes
  • Herring
  • Killer whales
  • Mammals

He got a bit confused at “killer whales” and knew he was off-track, but wasn’t sure what had gone wrong.

I think the lesson here is that even though people may primarily use the obvious tools and affordances presented to them, they will also make potentially incorrect assumptions and risk being led astray by well-intentioned sign-posts in the UI.

Some other kids’ misconceptions

One child apparently thought that the autosuggest was a list of answers to the query he was typing. That doesn’t always work perfectly:

But to be fair, it’s not immediately obvious that UX like “people also ask” (which does come with embedded answers where possible):

Is entirely different to related searches which are not necessarily even suggested sensible questions:

And finally, to end on a light-hearted anecdote from the research, probably my favourite story was the child (not mine!) who looked for both dolphins and information about the Vice President of the United States on the SpongeBob SquarePants website.

Presumably unsuccessfully, at least in the case of the VP’s birthday.

If you liked this post, check out the whole session from my recent SearchLove talk in San Diego (all you need to do is create a Distilled account to access it for free). You can also check out the slides from my presentation below. Enjoy!

WATCH THE VIDEO

What do dolphins eat? Lessons from how kids search was originally posted by Video And Blog Marketing

A Look Into the Latest Google Algorithm Update (July 2018)

A look into the latest Google Algorithm Update (July 2018)

Google is a search engine that always finds ways to innovate and optimize their services to improve the overall user experience and provide the best search results. This 2018, we have already seen numerous updates, from Google Images removing the ‘View Image’ option to Google launching an algorithm update last March that caused a fluctuation in rankings.

This past month, Google has once again launched a new set of algorithm updates that continue to optimize search. One of the latest updates is the Google Speed Update, which aims to implement mobile loading speed as a search ranking factor. Mobile SEO has been gaining more traction over the past year, which is in line with the growing number of mobile users using search on their devices. We have covered mobile optimization in detail previously, and with the updates rolling in, it is time to re-open the discussion. Here are some of the most important updates during June and July 2018, and how to optimize your website to comply to these changes.

Google Speed Update

The first update we will be discussing is the newly-launched Google Speed update, which adds mobile page loading speed as a ranking factor. Mobile search has seen a rise in the number of users and searches inquiries over the past few years, with more people having a preference for browsing using their mobile devices. This rise saw Google once again adapting to the current trend and crafting new updates that make search work more efficiently.

Page loading speed is an important ranking factor in Google, as it directly affects the user experience. If a website takes too much time to load, users will prefer to view a different website altogether, thus reducing your chances of helping your traffic grow. With the Google Speed update, loading speed is once again in the spotlight, this time on mobile. With mobile loading speed a ranking factor, mobile optimization is the key to making it all work better for your website.

We have covered in detail the important steps that need to be done to prepare for the Google Speed update, and now that it has arrived, your site would be well optimized, having solid load times. One important process that helps boost up your loading speed is by using website audit tools such as Google Lighthouse and Woorank, which allows you to review your website and see how you can improve loading speed. Caching tools also come in very handy, as they help load files much faster, boosting your loading speed significantly. AMP integration is also crucial, as this will

For those concerned with how will this update affect your website, Google stated that only the “slowest” websites will have their search rankings affected. Despite this statement, there is still a good chance that you might lose a good chunk of your traffic if you do not optimize accordingly.

Ranking Fluctuations in July

Along with the launch of Google Speed, there have been instances that rankings have been fluctuating over the course of a month, with a lot of SEO professionals thinking that another algorithm update might be underway and that Google is once again testing the waters. While these fluctuations are normal and have happened earlier in the year, it is still worth a look to make sure that your SERP rankings will not be permanently affected.

Accuranker Grump

It is important to note that a good number of tools are picking up these fluctuations. In Accuranker, the Grump rating for the past few days has ranged between Grumpy and Furious. This means that rankings have been going up and down during the past few weeks.

Accuranker Grump 2

SERPmetrics has also picked up these fluctuations as well, with the graph also bouncing up and down during the past few weeks.

SERPMetrics

This is another interesting development that is worth monitoring in the next few weeks, as Google tends to subtly test out these updates quietly before officially launching it. Whether this is another effect of the Google Speed update, or simply just a test, only Google can confirm our speculations. For the meantime, keeping watch on your rankings is a must this time of the year, as more updates will surely arrive soon.

Key Takeaway

Google algorithm updates are always big news, despite how simple some of these can be, as they have an impact on our rankings, and can be a sign that new strategies and techniques need to be developed in order to keep your rankings up. With Google continuing to push for more mobile-friendly updates, expect more changes in the upcoming weeks.

If you have questions about mobile optimization or SEO in general, leave a comment below and let’s talk.

A Look Into the Latest Google Algorithm Update (July 2018) was originally posted by Video And Blog Marketing

Product Pages Tested: How carefully pinpointing customer anxiety led to a 78% increase in conversion

The following research was first published in the MECLABS Quarterly Research Digest, July 2014.

 Product pages are a staple in nearly every business website in existence. Oftentimes, they represent the final hurdle before a prospect clicks “add to cart” or fills out your form. Therefore, if we can improve the performance of these key pages, we can see substantial increases in conversion and sales.

 

Figure 1.1

Look at the three pages in Figure 1.1. What do they have in common?

Granted, there could be multiple correct answers to this question. However, one similarity may have escaped your notice: anxiety. In every page, especially product pages, certain elements raise the anxiety level of the prospect. This should concern you for two very good reasons:

  1. In our experience, when we correct for anxiety, we see gains.
  2. The needed correction often involves only simple and small changes.

z

Figure 1.2

 

In the Marketingsherpa E-commerce Benchmark Study, we found ecommerce marketers are employing a variety of page elements that can be used to reduce anxiety (Figure 1.3).

Figure 1.3 – Page elements used by successful and unsuccessful ecommerce companies on product pages.
Anxiety-reducing elements highlighted.

We tested four of those minor elements to correct specific points of anxiety on the same page and to help us understand the interplay of anxiety and the corrections we make.

Experiment: Which anxiety correction had the biggest impact?

The experiment sought to improve the sales of e-books from a well-known retailer in the market. Our approach was to test four different variations of the product page against the control, with each treatment correcting for a different form of hypothesized anxiety:

  • Version A: Adjusting for anxiety regarding site security (Figure 1)
  • Version B: Adjusting for anxiety that the e-book would not be compatible  with their reading device (Figure 2.2)
  • Version C: Adjusting for anxiety that the e-book would not be of interest or value to them (Figure 2.3)
  • Version D: Adjusting for anxiety regarding the shipping time frame of the e-book (Figure 4)

What does your instinct tell you? Which, if any, of the corrections would most improve conversion?

The result: Version C was the winner, increasing conversion by 78%

After our complete analysis, we discovered three key principles as to why Version C was victorious, as well as what we can learn from the success of the other treatments.

How to correct for anxiety on product pages

Key Principle #1. Every element we tested on the page overcorrected some type of customer anxiety, with various elements performing more effectively than others.

It is crucial to note that while Version C produced the largest increase, each treatment page outperformed the control.  In other words, in every case where we took steps to alleviate customer anxiety, conversion went up. These results underscore the importance of this effort, as well as the relative ease with which gains can be achieved.

It is important to note the use of the term “overcorrect” here because anxiety is not always rational. You may know that flying in a plane is statistically safer than riding in your car, but, for many of us, our anxiety level is much higher in an airplane. Is it rational? No. Is it still very real? Yes. You may see no reason for concern about a given aspect of your page, but that does not mean anxiety is absent for customers.

Key Principle #2. The effectiveness of each corrective is directly related to how it matched the specific concern in the mind of the customer.

While all cases of anxiety correction produced lifts, one change impacted conversion significantly more than the others. Version C overcorrected for a concern that was most immediate to the prospect at the time. Therefore, it is crucial to discover the specific anxieties your customers are experiencing on your product pages. Among a plethora of options,  we have found some standard minor corrections you can make for specific anxieties:

Source                                                                             Correction

Product Quality Anxiety  ———————————> Satisfaction Guarantee

Product Reliability Anxiety  —————————–>  Customer Testimonials

Website or Form Security Anxiety  ——————->  Third-party Security Seals or Certificates

Price Anxiety  ————————————————>  Low-price Guarantee

Additionally, customer testimonials can be used to alleviate several different concerns. You want to choose testimonials that specifically deal with the point of anxiety the customer is experiencing (Figure 3.1).

Figure 3.1 – Examples of testimonials addressing specific points of customer anxiety

 

Key Principle #3. Location plays an important role. You can more effectively correct anxiety by moving a corrective element within close proximity to where the concern is experienced.

As in real estate, location is of utmost importance when correcting for anxiety on product pages. If you are correcting for form security concerns, you want the correction element right where the customer must click to submit the form. In Version C, we simply added a plot synopsis above the fold rather than farther down the page, and it led to the biggest jump in conversion. It’s not always about creating new elements, but instead, placing existing ones in a location that better serves the thought sequence of customers.

Overcorrecting for product page anxiety

Anxiety is lethal to product page conversion. It is always present, and it is not always rational.  By overcorrecting for predictable or discovered customer anxiety, you will empower more prospects to complete the sale.

The effectiveness of an anxiety corrective is dependent on two essential factors:

Specificity – How specific is the corrective to the source of anxiety?
Proximity – How close is the corrective to the moment of concern?

If you can identify the main cause of anxiety on the page and implement an overcorrection element in close proximity, you are on your way to higher conversion and more sales.

Related Resources

Landing Page Optimization: Addressing Customer Anxiety

MECLABS Research Catalog — Learn about other experiments performed in the MECLABS Laboratory

MECLABS methodology

Conversion Rate Optimization — Read how anxiety plays a role in building the “Ultimate Yes” to conversion

Online Testing: 6 Test Ideas To Optimize The Value Of Testimonials On Your Site

To learn more about anxiety and the factors that affect it, enroll in the MECLABS Landing Page Optimization Online Certification Course

Product Pages Tested: How carefully pinpointing customer anxiety led to a 78% increase in conversion was originally posted by Video And Blog Marketing

Playing detective: how to identify bad backlinks

I completed a backlink audit recently, and this is the post I wish I’d had when starting the tedious task of identifying the nasty links. Not all dodgy links are obvious, heck some are even near-impossible to find, especially when you have a spreadsheet containing thousands of them.

This is not a post about how to do a backlink audit from A-Z – that’s already been written about loads of times. Instead, I’m going to take you through how to identify patterns in your backlink data to quickly and accurately uncover spammy links.

I’ve written this post for the greater good of all SEOs, and yes, you’re welcome.

Wait – do I even need to do a backlink audit?

There has been some confusion since the last Penguin update as to whether or not SEOs even need to carry out backlink audits anymore. After all, Google has said that now they only devalue spam links as opposed to penalising the site receiving them, right?

Well, the short answer is: yes, you probably should continue to carry out backlink audits and update your disavow file. You can read more about this here and here.

Why can’t I just use an automated tool to find the bad links?

I know it’s tempting to get automated backlink tools such as Kerboo to do all the hard-lifting for you. Unfortunately, though, this isn’t a great idea.

In the backlink audit, I did recently, 93% of porn links were assigned a link risk of ‘neutral’ with a score of 500/1,000 (0 being the safest link and 1,000 being the riskiest). Links from the BBC also received a ‘neutral’ rating, with some getting a higher risk score than the porn links! Go figure.


Automated backlink tools can be super valuable; however, this is because of all the data they draw together into a single spreadsheet, as opposed to them being particularly accurate at rating the risk of links. To rely solely on their link risk metrics for your backlink audit is a quick ticket to trouble.

Is this guide relevant to my site?

This post is not a ‘one-size fits all’ strategy to a backlink audit, so please use your common sense. For example, below I recommend that root domains containing the word ‘loan’ are generally indicative of unscrupulous sites. However, if you’re doing a backlink audit for a financial services firm, then this generalisation is less likely to apply to you.

It’s up to you to think about the guidelines below in the context of the site you’re auditing and to adjust accordingly.

You will need

Before you start, you will need to have all your backlinks neatly assembled in a spreadsheet along with the following information:

  • URL (one example per linking root domain)
  • root domain
  • anchor text
  • citation flow (Majestic) or domain authority (Ahrefs or Moz)
  • trust flow (Majestic) or domain trust (Ahrefs or Moz)
  • backlinks
  • IP address
  • title
  • page language
  • link location
  • and anything else you can think of that could be useful

This article can bring you up to speed if you’re not sure how to assemble this data. Make sure to combine data from as many sources as possible, as different SEO tools will contain different information and you don’t want to miss anything! As I said earlier, I would also recommend Kerboo as one of your data sources, as it pulls a lot of the information you could want into one place.

How to spot the patterns

Fortunately for us, the bad guys almost always do their dirty work in bulk, which makes life easier for us good guys who inevitably have to clean up after them. It’s rare to find one dodgy directory submission or a single piece of spun content containing a paid link. This is a massive help – use it to your advantage!

Pivot Tables

I highly recommend creating a pivot table of your data so that you can see how many times an issue has occurred in your data set. This can help you to quickly spot patterns.


Above: spotting suspicious anchor text using a pivot table

For example, let’s say you’re doing a backlink audit for a clothing site. By pivoting for anchor text, you might be able to quickly spot that ‘buy cheap dresses’ appears several times. Given the commercial nature of this anchor text, it’s likely it could be spam. You could spot check some of these URLs to make sure, and if they’re consistently dodgy, you can reasonably assume the rest of the links with this anchor text are too.

Above: putting together a pivot table to spot anchor text frequencies (view large version of gif)

Word Clouds

Another thing I like to do is to dump my data into a word cloud generator. This is useful because it visualises the data (the bigger the word, the more times it appears in your dataset). It can help me to quickly catch something that looks like it shouldn’t be there.

Keeping on top of your data

Make sure you make a note as you work that explains why you’ve decided to disavow a set of links. It helps not just at the end when you’re reviewing your links, but will also be a big help when you come to spot patterns. It will also stop you from revisiting the same links multiple times and asking yourself ‘why did I decide these were bad links?’

Above: screenshot from my recent backlink audit with ‘action’ and ‘reason’ columns

Examples of common patterns to find bad backlinks

I’m now going to give you specific examples of bad links which you can use to find patterns in your data.

It’s not always a clear-cut answer as to whether a link is spam or not, however, the guidelines below should help guide you in the right direction.

When you’re unsure about a link, ask yourself: ‘if it wasn’t for SEO, would this link even exist?

Words to look for in the root domain or URL

X-rated words in the URL

You’ll immediately want to disavow (unless of course, these are relevant to your site) any x-rated links. These usually contain one of the following terms in their URL:

  • porn
  • xxx
  • sex (also sexy can result in some shady sites)
  • escort
  • dirty
  • adult
  • and any more dodgy words you can think of that relate to orgies, orgasms and other obscenities

Be careful not to accidentally disavow URLs where ‘sex’ is in the middle of a word – such as sussexhotels.com or essex.ac.uk. This will require some manual spot checking.

Root domain contains references to directories & listings

Next, you want to look for any URLs that indicate manipulative SEO link-building tactics. Directories are an obvious example of this, and while not all directories are bad (here is a good article on how to tell the difference), generally those created purely for link-building purposes contain the following words in the root domain:

  • ‘directory’ – especially ‘dir’ and ‘webdir’
  • ‘links’ – especially ‘weblinks’, ‘hotlinks’ or ‘toplinks’
  • ‘listings’

You might notice I’ve specifically said ‘root domain’ as opposed to ‘URL’ here. There is a reason for this: you might find lots of URLs in your dataset where ‘links’ is in the URL path. As a general rule, these are a lot less likely to be manipulative links. Compare http://www.lutterworthyouththeatreacademy.co.uk/links.html with http://www.speedylinks.uk. One of these is spam, and the other isn’t – can you spot the difference?

Root domain contains references to SEO

You’ll also find that if the root domain contains SEO or web-related terms, it’s likely it exists simply to serve the purpose of building links. Look out for the following words in the root domain:

  • ‘domain’
  • ‘search’
  • ‘seo’
  • ‘web’

Bear in mind that lots of sites have ‘search’ pages, so your best bet is to focus on the root domain for this to be an indication of anything suspect.

Content farms are another common feature of a poor backlink profile. Look for any domains that contain ‘article’.

Other dodgy root domains

The following keywords in the domain are usually indicative of dodgy link-building practices:

  • ‘cash’
  • ‘loan’
  • ‘com’ (such as com-com-com.com – yes, really)
  • ‘world’
  • ‘ads’

Root domain contains consonant or number clusters

Another obvious sign is any root domains which simply do not make sense. You’ll likely have lots of domains linking to your site consisting of bundles of consonants and letters, such as ‘1073wkcr.com’ or ‘a0924111232.freebbs.tw’. Watch out for domains like these, as more often than not they are low quality.

You can easily find URLs like this by sorting your root domain column from A-Z. You will find that:

  • any domain starting with a number will appear at the top of your list.
  • scrolling to the bottom to letters x, y and z usually throws up lots of domains with consonant clusters that do not make sense.

The ccTLD is uncommon

Uncommon ccTLDs are usually indicative of dodgy sites. Any site worth its salt will try and obtain the .com, .net, .org, .edu or relevant country ccTLD for its domain name. The less common ccTLDs are an indication of a lower quality site and those examples I found in my most recent backlink audit which indicated spammy sites were:

  • .biz
  • .casino
  • .clothing
  • .ooo
  • .properties, etc

Looking at titles for further clues

When the domain name or URL isn’t particularly insightful, the page title is the next place to look. Look out for the same keywords listed above, as well as the following phrases:

  • ‘most visited web pages’
  • ‘reciprocal links’
  • ‘link partner’
  • ‘link exchange’
  • ‘seo friendly’

Another clue is to find any site titles that are completed unrelated to the niche of your site. Titles that contain commercial terms are particularly suspect, such as 

  • ‘louis vuitton belts’
  • ‘nike shoes’

As I mentioned before, bad backlinks often operate in bulk, and there’s nothing like a load of duplicate titles to lead you hot on the heels of a group of spammy URLs.

What can anchor text tell us?

Is it keyword-heavy?

A popular SEO tactic in the pre-Penguin days was to link to your site with keyword-heavy or commercial anchor text, such as ‘cheap red dresses’.  Make sure to put together a pivot table of your anchor text so you can quickly scan for any recurring anchor text that looks suspiciously well-optimised and check out these links to see if they’re legit – they probably aren’t.

Does it make sense?

In addition, any anchor text that simply doesn’t make any sense or is completely unrelated to the site you’re auditing is highly likely to be low quality.

Is the language consistent with the rest of the page?

Finally, any anchor text that is in a different language to the rest of the content on the page is likely to be a paid link. You can use the ‘language’ column (provided by Ahrefs and Kerboo) to see what language the page is in, and you can compare this to the language of the anchor text of your links. Anywhere where there is a mismatch is likely to be suspicious.

Duplicate root IP address

Pivot your data to see if there are several with the same IP address. If there is a block of URLs that share the same IP address and one of these is spammy, it could be likely that the rest are too.

Make sure to do a manual spot check of the sites to make sure you’re not disavowing anything harmless. For example, sites hosted at blogspot.com and wordpress.com are commonly hosted at the same IP address, and many of these will be harmless.

Where on the page is the link located?

In many backlink reports, there’s a column which tells you where on the page the link is located. In Kerboo, this column is called ‘link section’, and it’s another nifty tool for us to use in our hunt for dodgy links. Filter this column for keywords contained in the footer and sidebar to see if there are any which look suspicious on opening the page.

Footer and sidebar links are prime locations for dodgy backlinks. Why? Because these are site-wide, they are often targeted for paid link placements as the recipient of the link can often benefit from the most link equity in this way.

In addition, if the link is providing no value to users on the site (for example, if it’s completely unrelated to the site content, which is likely if it’s a paid link) then the footer is a useful place to essentially ‘hide’ the link from users while still providing link equity to the recipient.

Where is the link pointing to?

In the ‘link to’ column, look out for links pointing to the ‘money pages’ on your site – these are any pages which are revenue-drivers or particularly important for other reasons, such as product pages or sign-up pages.

It’s natural in a backlink profile to have the majority of links pointing to your homepage; this is where most people will link to by default. It’s much harder to build links to pages deeper in a site, especially product pages, as it’s not particularly natural for people to link here.

By glancing an eye over links which point to money pages, it’s likely you could spot a few suspicious links which have been previously built to help boost the rankings of important pages on your site.

Taking things to the next level

All the tips I’ve shared with you so far have involved mining data that is easily accessible to you in your backlink spreadsheet – things such as root domain, URL, page title and anchor text.

To take your backlink audit up a level, it’s time to get savvy. This is where Screaming Frog comes in.

Using Custom Search to spot link directories

You know how earlier we mentioned that not all directories are bad? Well, an easy way to spot if a directory exists solely for link-building purposes is to see if the page contains phrases such as ‘submit link’, ‘link exchange’ or ‘add your site’.

These telltale phrases will not necessarily be in the URL or page title of your link, so this is why it’s necessary to take things up a step.

To find pages which contain these terms, you can run a crawl of your backlink URLs using the Screaming Frog Custom Search feature.

Above: using Screaming Frog ‘Custom Search’ to find web pages containing suspicious text

Once the crawl is finished, you can then download the URLs that contain the phrases above. These will most likely be some obvious link directories that you’ll want to disavow pretty sharpish.

Using Custom Search to spot spun content

The Screaming Frog custom search feature isn’t just useful for finding directory links. This is where you really need to put on your detective hat and to have a good think of any patterns you’ve noticed so far in your backlink audit.

When I did my audit recently, I noticed a recurring theme with some of the paid links. There were links to other sites with commercial anchor text that kept appearing alongside the link to the site I was auditing. This was a piece of spun content that had been copied and pasted across multiple sites and forums, and whoever had done the work was clearly being lazy, lumping a load of unrelated links together in one paragraph.

Apart from the fact the text made no sense whatsoever, the anchor text of these other links was extremely commercial: ‘cheap nike free run 2 for men’ and ‘chanel outlet UK’ where a recurring theme.

Above: example of spun content that appeared in my recent backlink audit

I’d tried to find a pattern in the URLs or titles of these pages, but it was a bit hit and miss. It was then that I realised I could do what I had done to find the directory links – Screaming Frog custom search.

I, therefore, carried out a Screaming Frog crawl that looked for recurring anchor text such as ‘cheap nike’ and ‘chanel outlet’ to identify any URLs that I hadn’t yet uncovered. It was extremely useful and allowed me to identify some URLs that up to that point I had been unable to identify from the data in my spreadsheet alone.

To wrap up

If you’ve made it this far, congratulations! I appreciate this post was a lot of writing, but I hope it’s really helped you to dig out any dodgy links that were lurking under the surface.

If there’s one thing to take away, it’s to look for any patterns or consistencies in the dodgy links that you find, and to then use these to dig out the less obvious links.

Do you have certain criteria that you find helpful when identifying bad backlinks? Comment below!

Playing detective: how to identify bad backlinks was originally posted by Video And Blog Marketing

How Hashtags on YouTube Can Improve Searchability

How Hashtags on YouTube Can Improve Searchability

With Google constantly updating their search algorithms and services on a monthly basis, it is only fitting that their Google affiliate YouTube also launch their own set of updates as well. Being the second biggest search engine in the world, YouTube also has the largest online video database in the world, with billions of hours’ worth of videos to watch.

Along with being the largest video platform, it is also one of the highest earning websites as well, with a huge number of users earning through their monetized videos. With the overwhelmingly large amount of videos in the site, YouTube also has its own ways to optimize search and improve visibility. One of the newest ways to search videos on YouTube is through the use of hashtags. The use of hashtags has already been seen on social media sites like Facebook, Twitter, and Instagram, and has helped create some of the most successful and viral social media posts and campaigns.

With this new feature, let’s see how hashtags help expand YouTube search and help you find the right videos.

How Hashtags Work

Ever since hashtags were introduced on Twitter in 2009, it has become an important element in social media posts, as it provides the option for users to be able to track various posts and events with similar hashtags. Putting a hashtag on a post will add a hyperlink that allows users to click on it and see the posts. Here is how hashtags look like on their respective platforms.

This is how it looks like on Twitter:

Hashtag Twitter

This is how hashtags can be viewed on Facebook:

Hashtag Facebook

And this is how hashtags look like on Instagram:

Hashtag Instagram

Hashtags on Facebook, Twitter, and Instagram work in similar ways, as they gather up all of the related posts in a single place.

Twitter Trends

For Twitter, hashtags are also used to track down trending topics and posts, which can help you see which ones have gone viral in certain parts of the world. For Instagram, having multiple hashtags helps posts become more visible to users, as they will be able to view them in a single place.

Hashtag YouTube

When it comes to hashtags on YouTube, hashtags are used as another method to be able to search for various videos. For example, if I want to watch a video about the latest Nintendo news and announcements, I have to use #Nintendo to look for related results. Like the three social media platforms, YouTube uses hashtags to make related content easier to view in a single location, allowing you to find the best content for your keywords.

How to add hashtags to your videos

Hashtag YouTube Description

Adding hashtags on YouTube is as easy as adding a video description and video title. All you have to do is to add the hashtags on the video description itself. After uploading the video, you will be able to see the hashtags above the video. Based on what I have seen in numerous videos that have used hashtags, I can only see at least three hashtags above the video title. It is important to note that you can only add a maximum of 15 hashtags in a single video to prevent getting penalized for over-tagging.

Will this affect YouTube SEO?

All in all, this is a simple and straightforward process that will become another important part of your YouTube SEO. While the change only requires you to add hashtags for searchability, choosing the right ones now become a crucial element. One of the main reasons why Hashtags are added into the platform is because YouTube wants users to be able to avoid misleading and offensive content, as per their new policies. This means that users will be able to access more quality, informative, and authoritative content that fits their preferences.

While hashtags are still a new feature on YouTube, it is important that you update your older videos to further improve their searchability, and make sure that these hashtags truly reflect the content that users would expect after typing in that hashtag.

Key Takeaway

Hashtags have become an important element in social media, and with YouTube finally adding them into their platform, this adds another degree of searchability that users and brands will find very useful in the future. While it may be a simple update at a glance, this adds another degree of search in the world’s second largest search engine.

If you have questions and inquiries about YouTube SEO and SEO in general, leave a comment below and let’s talk.

How Hashtags on YouTube Can Improve Searchability was originally posted by Video And Blog Marketing