Highlight links internal

Posted on by

Browseo is a wonderful tool for browsing pages through the eyes of a search engine. When it comes to analyzing site structure and other factors that pertain to an entire website, Screaming Frog is the tool of choice for many to use alongside Browseo. Both tools together are a perfect match.

Besides Browseo, Screaming Frog is one of the most powerful tools a technical SEO has in his toolbox. That said, the recent additions to the software have made it a "must have" for every SEO. Out of necessity I built a similar spider back in the 90's so I know how hard it is to build a spider to evaluate site architecture and structure. The new features take "the Frog" from being a mainstay in the SEO toolbox to a tool that webmasters should be looking at to monitor site health and stay ahead of what can become very large numbers, very fast, in Google Webmaster Tools.

The Sitemap Generator

Yeah! I know you are thinking "big deal there are a bunch of free tools that do that very well!". So webmasters generate it and adjust as content is added. Yes that insures indexing but does it enable you to identify new content or changed content? Likely not. Whereas Screaming Frog includes the last modified date from the header and from that you can learn:

  1. what is new on the site
  2. what has been changed
  3. when these events occured

Last modified is a very important value in the header as Google for sure uses it to partly determine what is fresh and when indexing refreshes should occur. Since Google has been known to favor freshness a few SEO's and spameisters have made their last modified dynamic. Sorry folks, Google is not a fencepost so I would be very, very careful with that stuff. So we have this XML sitemap with last modified how do we use this to our advantage?

Simple! Open Excel import the xml file and then sort (descending) on the last modified field. Voila! At the top of the list are the new and changed files! I suggest a weekly indexing and saving your last indexing for comparison. Again, When your client screws up and potentially tanks their rankings by say .... I don't know... setting the "discourage Search Engines..." setting in WordPress...  Don't laugh I've seen it happen more than once! With weekly indexing you may even be able to catch it before the Search Engines do.


This feature is amazing for testing how spiders react! Testing robots.txt and .htaccess is now as simple as adding the list of URLs you want to test into Screaming Frog! The first thing I do for site migrations or redevelopments is index the site just before the migration/changes take place. That way I have lists to work from for my robots.txt and .htaccess files. Once those are written and the development/migration is complete I use the list feature in Screaming Frog to test the links, robots.txt and redirections.


This is very cool because you can get an idea in real seconds which makes sense whereas Google tool uses some cockameme grading thing. However the Google tool is very useful in figuring out exactly what page elements are responsible for the slow load speed. I like to make decisions on "real numbers" not some formulas. How about you?

So, what do you do when you see a need for speed? Use Screaming Frog to identify the problematic pages then you run the pages that need improvements through the Google tool to see the best way to improve the load speed. This saves you time and enables you to make more informed decisions about what pages to implement changes on because they are based on real numbers that you can understand and know exactly how it affects the user experience.


There is lots of other new features in Screaming Frog I strongly suggest you learn what data it collects and as you go through that the ways that it can benefit you will become endless. That and the the never ending additions to this program like Video and Media sitemaps means this will be a tool you'll rely on for many years so learning what it does and how will make you that much better as an SEO or webmaster!

Terry Van Horne


Partner, SEO Training Dojo.   Terry Van Horne has been developing and marketing websites since the early 90's in various marketing and development positions including working as internet marketing manager for one of Canada's largest real estate developers; SEO for an award winning real estate search engine and marketing manager for ecommerce stores in the apparel and musical instruments. In 2007 he developed a YouTube Marketing Strategy and to date those 300+ videos have received over 26,000,000+ downloads . He is currently a partner with David Harry in Reliable SEO, a Search and Social Media Marketing Agency, and the award winning SEO Training Dojo, a learning community as well as three other marketing and industry news sites. Terry is Director Of SEO Dojo Radio and co-hosts 2 hour long podcasts "Search Geeks Speak," an hour with luminaries and up and comers in Search Engine Marketing as well as "The Regulators," an hour of Industry News. Terry founded SeoPros.org, the first consumer advocacy organization for purchasers of search engine optimization and is currently a Director of the NFP organization OSEOP that grew out of it. Follow him on Twitter.

45 Responses to “Tool Review: 3 Tips for Using Screaming Frog’s New Features”

  1. Ryan Dozier says:

    I’ve just discovered this tool and already can see the benefits it has.

  2. Doris says:

    I think it is not a disadvantage if you pay $ 99 a year for this extensive software. Other professional SEO tools cost 50, 100 or more $ every month and the Screamingfrog offers for this price incredibly many ways.

  3. Agen Sbobet says:

    You may post on the professional credentials for the blog owner. You could express it’s outstanding. Your blog experience can springboard your click through.

  4. Taruhan Bola says:

    This is my first time i visit here. I found so many entertaining stuff in your blog, especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the leisure here! Keep up the good work. I have been meaning to write something like this on my website and you have given me an idea.

  5. Thanks so much with this fantastic new web site. I’m very fired up to show it to anyone. It makes me so satisfied your vast understanding and wisdom have a new channel for trying into the world. Jadwal Liga Spanyol Klasemen Liga Spanyol

  6. jassica says:

    The surface isn’t slippery and you can easily clean too. The rubberized grass matting is actually safe to become used along with areas which have kids. RUBBER GRASS MATTING

  7. petaruh says:

    play with nice agent sbobet or ibcbet and get the best odds handicap from your agent trusted

  8. vacation resorts says:

    thanks for the sharing, here it’s awesome blog about hawaii all inclusive resorts if you need to hawaii, you must read first !

  9. Car finance or even auto financing may be the phrase utilized when a person wants to buy a vehicle but cannot provide the complete required quantity of the automobile upfront. Within an auto financial process, the person purchasing the automobile might first submit an application for auto funding. auto-forex.net

  10. autorentool.net says:

    Simply just how much can this type of software program which can be auto prosperity maker cost? The tougher accurate question for you personally is simply just how much do all of us desire the buying price of many becoming? Nicely, whilst consumer, we wish to check out the plan also you want to try away auto prosperity maker In the event that it truly brings cash, you need to can easily make money using it after which we’re prepared to spend excellent price for your too. autorentool.net

  11. switchome.net says:

    Your own economy is within complete control by making use of few fundamental steps. You’ll be more comfy in residing in healthier atmosphere. ‘House within order’ provides you with good satisfaction. When a person build your house you may build this more power efficiently. Show homes tend to be always appealing and amazing. switchome.net

  12. homebuildingandloan.net says:

    Because homeowners have become more mindful about improving the look of their house, home enhancement industry offers undergone an enormous revolution. Several creating firms are actually bringing clean home renovation tips to the desk. Some companies also have gone on the internet to plunge into this particular new rising market. homebuildingandloan.net

  13. dogtrainingphoenix.net says:

    Occasionally, your the next door neighbors children have an interest in taking care of your home pets. Yet this is probably not the smartest choice, as kids could be reckless as well as ignore fundamental needs of the animal Because of this, protection from the overall health of your own pets ought to be handed more than in compensated professionals, as you’ll get calmness and also the serenity associated with mind when you are not obtainable. dogtrainingphoenix.net

  14. collegecreditcareersnetwork.net says:

    Profession development, however, is often viewed as more of the big image view. While advice might be something a person seek to satisfy an instant need, career improvement is something which unfolds as time passes. The book definition associated with development, because noted through Webster is actually, “the behave, process, or consequence of developing. collegecreditcareersnetwork.net

  15. constructioncareeraustin.net says:

    Before you decide to make any kind of decision associated with your career you need to assess your self first. Since the inherent capability, skills as well as values tend to be most helpful items to make profession choices. You make use of self checks tools quite simply we may say work test for any self evaluation test. A profession test allows you to making decisions for the career. Additionally, it creates a suitable list associated with occupation as well as gathers correct information for the career. constructioncareeraustin.net

  16. technews404.net says:

    Profession advice nowadays comes in most size and shapes. It’s simple to get “lost within the lingo” however understanding a few basic definitions might help enable you to get the correct support to your requirements. To place it simply, whether you’ll need career guidance or profession development depends upon where you stand on your job journey. technews404.net

  17. hay day says:

    This blog is so nice to me. I will keep on coming here again and again. Visit my link as well..

  18. Are searching for free itunes codes then you should visit at given link atlast once.

  19. GA is now integrated in latest version, great tool.
    Thanks for sharing.

  20. Terrific post however , I was wanting to know if you could write a lite more on this
    topic? Obat Hemoptisis Cara Menyembuhkan Pleuritis
    I’d be very thankful if you could elaborate a little bit further. Thanks !
    Cara Menurunkan Kolesterol Cara Mengobati Radang Sendi Cara Menyembuhkan Flu dan Batuk

  21. sabir234 says:

    i want all the codes of google at https://freegoogleplaycode.wordpress.com/

  22. wallens says:

    I can see that you are an expert at your field! I am launching a website soon, and your information will be very useful for me.. Thanks for all your help and wishing you all the success in your business.

  23. wallens says:

    This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information…
    amasuite 4.0 review

  24. nona says:

    Excellent blog you have here but I was curious about if you knew of any community forums that cover the same topics talked about in this article? I’d really like to be a part of online community where I can get advice from other experienced individuals that share the same interest. If you have any suggestions, please let me know. Appreciate it softonice | canon support | drivers and software

  25. Akash Sharma says:

    Thanks for Nice and Informative Post. This article is really contains lot more information about This Topic. –
    Packers and Movers Hyderabad
    Packers and Movers Mumbai
    Packers and Movers Chennai
    Packers and Movers Noida

  26. Shubham says:

    Thanks for this cool post on your blog really looking forward to read more here
    a tech blog techin idea for free

  27. Prediksi Bola dan Berita bola paling lengkap hanya di Prediksi Bola

  28. Mukesh says:

    Thanks for sharing interesting message for supporters because those people always wait for this.


  29. Interesting and amazing how your post is! It Is Useful and helpful for me That I like it very much, and I am looking forward to Hearing from your next..

  30. I have bookmarked you to check out new stuff of your blog. A must read!!!!

  31. You have a very inspiring way of exploring and sharing your thoughts.
    american home warranty reviews

  32. We are really grateful for your blog post. Great work !!
    texas home warranty

  33. moha amad says:

    I was about to say something on this topic. But now i can see that everything on this topic is very amazing and mind blowing, so i have nothing to say here. I am just going through all the topics and being appreciated. Thanks for sharing.
    Software and Printer drivers Download

Leave a Reply