To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.[47]


Firstly, really think about what your audience is interested in and what their needs are. As SUCCESS agency CEO, Avin Kline, states, “It’s so easy to forget, but the heart of increasing user engagement is to put yourself in their shoes and add undeniable value to the user. Keep in mind, what marketers think is valuable and what users think is valuable are often two different things.”
This information hits the mark. “If you want your content to go viral, write content that influencers in your niche will want to share.” I love the information about share triggers too. I’m wondering, though, if you could share your insights on how influencers manage to build such vast followings. At some point, they had to start without the support of other influencers. It would seem that they found a way to take their passion directly to a “ready” world. Excellent insights. Thanks for sharing.
Laura,Great post.  This touches something I wish more SEOs practiced: conversion optimization. I think most SEOs think of what they do as a service for, instead of a partnership with clients.  The end result should never be raw traffic, but value obtained through targeted, CONVERTING traffic.You make excellent points about market research, product input, content creation, and other functions many SEOs and SEMs neglect.More and more SEO providers focus only on assembly line basics and worn out techniques instead of challenging themsleves to learn product marketing, usability, and conversion optimization.Your advice on market research is extremely valuable.Great start to a promising series.  I look forward to more!

Awesome tips Brian. Always enjoy your posts. My question is, how can I boost traffic significantly if my keyword has pretty low search volume (around 100 monthly searches based on keyword planner)? I’ve been trying to expand my keyword list to include broader terms like “customer experience” but as you know that is super competitive. Do you have any suggestions for me? Thanks in advance.

There were some great tips in this article. I notice that many people make the mistake of making too many distracting images in the header and the sidebar which can quickly turn people off content. I particularly dislike google ads anchored in the centre of a piece of text. I understand that people want to make a revenue for ads but there are right ways and wrong ways of going about this. The writing part of the content is the important part, why would you take a dump on it by pouring a load of conflicting media in the sides?
All the content published on the MyThemeShop.com domain including images, site content published on the showcase and on the blog, belongs to MyThemeShop and is under copyright. Any reproduction of the site content has to be authorized and distinctly referenced back to the source. Written consent of MyThemeShop is required before the MyThemeShop website is used or exploited for any commercial and non-private purpose. Though the content published on demo sites is non-exclusive and is not copyrighted.​
Use your keyword list to determine how many different pillar pages you should create. Ultimately, the number of topics for which you create pillar pages should coincide with how many different products, offerings, and locations your business has. This will make it much easier for your prospects and customers to find you in search engines no matter what keywords they use.
In March 2006, KinderStart filed a lawsuit against Google over search engine rankings. KinderStart's website was removed from Google's index prior to the lawsuit, and the amount of traffic to the site dropped by 70%. On March 16, 2007, the United States District Court for the Northern District of California (San Jose Division) dismissed KinderStart's complaint without leave to amend, and partially granted Google's motion for Rule 11 sanctions against KinderStart's attorney, requiring him to pay part of Google's legal expenses.[70][71]
Search engines find and catalog web pages through spidering (also known as webcrawling) software. Spidering software "crawls" through the internet and grabs information from websites which is used to build search engine indexes. Unfortunately, not all search engine spidering software works the same way, so what gives a page a high ranking on one search engine may not necessarily give it a high ranking on another. Note that rather than waiting for a search engine to discover a newly created page, web designers can submit the page directly to search engines for cataloging.
Inclusion in Google's search results is free and easy; you don't even need to submit your site to Google. Google is a fully automated search engine that uses web crawlers to explore the web constantly, looking for sites to add to our index. In fact, the vast majority of sites listed in our results aren't manually submitted for inclusion, but found and added automatically when we crawl the web. Learn how Google discovers, crawls, and serves web pages.3
Content gaps – make an inventory of the site’s key content assets, are they lacking any foundational/cornerstone content pieces, non-existent content types, or relevant topic areas that haven’t been covered? What topics or content are missing from your competitors? Can you beat your competitors’ information-rich content assets? Useful guides on Content Gap Analysis:

See the screenshot below for some of the sections for specific recommendations that you can add which will provide the meat of the document. Keep in mind this is a very flexible document – add recommendations that make sense (for example you may not always have specific design considerations for a project). Remember, it will be different every time you do it.


Tip: Along with delicious I search on scoop.it for similar opportunities. If they liked an article related to a year.. say 2013 and you update the resource to 2014 chances are they’ll share it. Kind of a twist on your delicious + sky scraper technique. You don’t even have to make the content much different or better, just updated! Got some fantastic links recently because of it.
Usually Search-engines automatically crawl your articles if it is high-quality but you should also try to submit your blog to search engines like Google, Bing, and Ask etc. Search engines like Google have already simplified the way of submitting your content. Google Webmaster Tools makes it easy for every webmaster to get their website crawled faster.

Ask a marketer or business owner what they’d like most in the world, and they’ll probably tell you “more customers.” What often comes after customers on a business’ wish list? More traffic to their site. There are many ways you can increase traffic on your website, and in today’s post, we’re going to look at 25 of them, including several ways to boost site traffic for FREE.
Hey Brian, love your site + content. Really awesome stuff! I have a question about dead link building on Wikipedia. I actually got a “user talk” message from someone moderating a Wikipedia page I replaced a dead link on. They claimed that “Wikipedia uses nofollow tags” so “additions of links to Wikipedia will not alter search engine rankings.” Any thoughts here?

Hack #1: Hook readers in from the beginning. People have low attention spans. If you don’t have a compelling “hook” at the beginning of your blogs, people will click off in seconds. You can hook them in by teasing the benefits of the article (see the intro to this article for example!), telling a story, or stating a common problem that your audience faces.
While with search advertising, you’re paying to show up in the top spot for relevant searches, with social media advertising you are paying to show up in relevant feeds. With both forms of advertising, you can specify the type of audience in front of which you’d like to appear, but with more psychographic data, social media offers superb targeting.

Search engines use complex mathematical algorithms to interpret which websites a user seeks. In this diagram, if each bubble represents a website, programs sometimes called spiders examine which sites link to which other sites, with arrows representing these links. Websites getting more inbound links, or stronger links, are presumed to be more important and what the user is searching for. In this example, since website B is the recipient of numerous inbound links, it ranks more highly in a web search. And the links "carry through", such that website C, even though it only has one inbound link, has an inbound link from a highly popular site (B) while site E does not. Note: Percentages are rounded.
×