The Vital Crawl of a Technical SEO with Rachel Anderson

We got a bit technical on today's Agency Ahead podcast. Okay, okay, we got real technical, because our guest was Rachel Anderson, a technical SEO analyst at DeepCrawl, a cool tool that allows you to understand all the technical aspects of your website and how they might ultimately impact SEO performance.

Tune in if you're looking to dive deeper into the world of technical SEO or just want a few insights into what's ahead.

The highlights:

[1:19] Rachel's role at DeepCrawl.

[2:26] Common site problems.

[3:39] Web core vitals.

[6:38] Presenting competitor data to clients.

[7:59] Learning Google Data Studios.

[10:34] Rachel's approach to building out reports.

[13:13] Small site technical audits: what do they take?

[16:41] What to consider before adding tech SEO services to your agency.

[20:22] Rachel's causes.

The insights:

Rachel's role at DeepCrawl

Rachel is a member of the Professional Services Team. 

"There's a group of about 10 of us. We have actual clients we work with. Obviously, DeepCrawl is a software provider, and you can just use the software, but we work with a lot of enterprise clients to help them get the most out of the software, and then go beyond it. We'll work with them on specific projects."

What kinds of projects?

"We do monthly crawl reviews to make sure there have been no big issues with their sites, and then we collaborate with different teams at DeepCrawl to roll out new features. This week we actually rolled out some really cool Schema validation features and performance metrics. So we got to have a sneak peek and input into that as well."

Garrett asked what big common themes Rachel sees when someone comes to her asking for help.

"Oh my gosh. Nobody can get their canonical tags right. Sitemaps are an issue. There are so many URLs. There's so much more to go on. I could go on for a very long time."

The growing importance of Google’s Web Core Vitals

Web Core Vitals are a set of performance metrics that Google intends to add to the algorithm in 2021. The vital statistics are Largest Contentful Paint (LCP) or the time it takes the largest image or text block in a user's viewpoint to be fully rendered after page load, First Input Delay (FID), the amount of time it takes for a page to be ready for user interactivity, and Cumulative Layout Shift (CLS), the amount text shifts around on a page as it loads.

Rachel says most of her clients are very interested in that data. The first thing she does is find competitor stats. 

"I don't really care how a local business site is ranking up against The New York Times' performance metrics. I care how The New York Times is doing vs. The Washington Post, side by side."

She says they use Google's user data to get that information, on a domain level.

"That's usually not enough. Like, okay, cool, so CLS is bad, but where? Is it bad everywhere? Is it bad on particular templates? With DeepCrawl we can look at CLS and LCP." 

"We can incorporate CLS and LCP into a crawl and get the metric for each individual URL. Then in our tool, we can actually set up sitemaps based on URL type. I can set up a category segment, a product segment, a blog segment, and look across those URLs to see what sections of my site are having performance issues.

That's been really helpful for clients to identify the specific sections of their site that might need a bit more work. We can actually crawl their competitor's site with those same metrics and set up similar segments. So we can say, yeah, your product pages are actually doing a lot better than your competitors." 

Of course, sometimes they're doing worse than their competitors.

"Then that's a different conversation."

Rachel suggests pulling the information into Google Data Studio. 

"I can now pull those metrics with the segmentation straight into Data Studio, so now I can actually have a client with their competitor with their segments side-by-side. A lot of clients have found it really effective for getting buy-in internally. 

Because they can then take that to their boss on the dev team and be like, hey look, it's the product pages that are having this cumulative layout problem and it's because this banner at the bottom is causing weird things to happen. 

They can actually have that conversation."

The process of self-educating on Google Data Studio

Garrett asked Rachel to describe how she learned the ins and outs of Google Data Studio.

"Back in 2018, I knew it was a thing and I was interested in it.

 Then my agency at the time sold a project to a client that was helping them set up some Google Analytics reports, but we didn't realize at the time of selling it that it was actually different analytics accounts. 

So I had to figure out how to use Google Data Studio for that project."

She says it was a lot of self-teaching because at the time there weren't really a lot of good resources online.

She says that these days Google has a Google Data Studio course that's really helpful. 

VIDEO: https://www.youtube.com/watch?v=4lPmu8fsOHc&feature=youtu.be

"I went through it right after it was released. I found it was really helpful for learning how to share reports and the data sources with people. As far as creating reports I didn't think it was useful at the time. I think now it's more useful."

She says she recommends people find reports other people have made open source. 

"Then manipulate them. Deep Crawl has an example template you can start with. Then you can mix and match metrics." 

Rachel's approach to building out reports

"I usually start with a table. I'll get a feel for what my metrics and dimensions actually are.

What can I do with this? How are these things related?

From there, once I understand what I have, I'll start building on different kinds of charts and tables. Usually if I can put a map in there I'm going to do that somehow, even if it's not necessary, because I love the Google Maps integration for Data Studio."

Once she's got a lot of raw data in one place, "I make some more adjustment, focus in on the point of the report, and pull in another SEO to come look at it. What is this telling you? Am I missing things here? Is this useful?" 

Small site technical audits: What do they take?

Garrett brought up a discussion that Emily Brady was having on SEO Twitter about technical SEO audits and the difference between a small site and a big site.

There was a discussion about technical SEO audits, and a lot of criticism saying you shouldn't charge a lot for the technical audit of a small site because it's so simple. 

Both Rachel and Emily felt strongly this wasn't the case, that there's a lot to a small site technical SEO audit that brings value. 

Rachel weighs in.

"Not all small sites belong to SMBs. There are some really large brands that I would consider enterprise that have small sites. They're not eCommerce. They're just a really large brand. 

They don't need a lot of pages. But they still manage to screw up technical SEO. There's plenty to look at."

She points out that she's audited sites with everything from a few pages to 4 million URLs.

"You're looking for different things. A lot of the small sites I've done audits for, nobody has ever claimed their Google Search Console account. There's plenty to do. 

I haven't had an enterprise-level client where they haven't claimed Google Search Console. The issues are different. But they still need technical SEO just like large sites."

She points out that a lot of times they've never even looked at their technical SEO. 

"So you're starting from nothing. 

And usually, nothing is things like – a robots file that's blocking the entire site, or entire sections of the site. Or they don't have any site maps. Or no canonical tags, ever. Or you know, the slash and no slash version of URLs may both be in 200 status. 

There are always problems. Just because it's a small site doesn't mean they don't need technical SEO."

She points out that for a small site she's going to spend a lot of time looking at nontechnical elements as well. 

"If you were going to do a Google Data Studio report I would totally recommend pulling in Search Console data around keywords. You can actually dig really deep into individual pages and their performance with small sites. I think in that case you could totally do a dashboard. The thing with big sites is we're looking for trends. We're looking for page template trends. Things that will really move the needle. 

We can't focus a ton of attention on individual pages unless they're high revenue pages."

What to consider before adding Technical SEO services to your agency offering

"First of all, you're going to need a good crawler," Rachel warns.

"At my last agency I remember getting the first eCommerce clients, and we were trying to crawl with Screaming Frog. I get clients, we get sales prospects like this all the time: I'm trying to crawl my 1 million URL site with Screaming Frog, but my computer just can't handle it."

Rachel hastens to say that she loves Screaming Frog.

"But it's not a proper tool for this. So first of all invest in a good crawler. I'm not saying that just because I work at Deep Crawl. It's because it saves you so much time and heartache and stress just to have a good crawler."

She points out some of the issues a good crawler will help find, like non-200 status pages in sitemaps, or the existence of orphan pages.

"I think gaining familiarity with the reports that crawlers offer you is a great starting point. And if you don't have much experience with crawling large sites, just start crawling parts of sites and look at the type of issues you see."

What’s your right now cause?

While it isn't a cause, Rachel does suggest women who are in SEO should be on the Women in Tech SEO Slack Channel. 

"That place is incredible. You can ask a question and will get an answer.

Sometimes on Twitter if you put a question out there, you'll get answers, and they'll be in a: you must be very dumb for not knowing this kind of way. The responses on Women in Tech SEO are always really kind and uplifting. 

We're all learning together. This is something I can help you with. I love that about the community."

A few weeks ago Women in Tech SEO started a mentor program that's been a huge success as well.

The other thing Rachel wanted our listeners to pay attention to was mutual aid societies. 

"There's Mutual Aid groups all over the country. They're helping to distribute food, medicine, and other necessities." 

She was able to give tomatoes and chicken eggs from her garden, but they also take financial contributions.

"I love it. The community is helping each other. They'll put out requests for things that are needed by community members, and then community members can step up and fill that need."

Her local one is MAD RVA (Mutual Aid Distribution Richmond VA) but they have Mutual Aid societies all over the nation.

Connect with Rachel Anderson

Want more of Rachel's insights? She's active and ready to chat on Twitter.

Website
Twitter
LinkedIn