Domain Verification for Vertex AI Search

Technical documentation on how domain verification works for website indexing

Overview

Domain verification is a security requirement by Google Cloud Platform (GCP) for Vertex AI Search datastores. It ensures that only authorized website owners can index their content for AI-powered search and content generation.

When you add your website to Antwork, we create a Vertex AI Search datastore that needs to verify domain ownership before it can begin indexing your pages. This verification process protects your content and ensures compliance with Google's security policies.

Why Verification is Required

  • Security: Prevents unauthorized indexing of websites you don't own
  • Content Protection: Ensures only legitimate website owners can use their content for AI training
  • Compliance: Meets Google Cloud Platform security requirements
  • Data Integrity: Guarantees that indexed content comes from verified sources

How Verification Works

Step 1: Datastore Creation

When you add your website URL in Antwork, our system automatically creates a Vertex AI Search datastore configured for advanced site search (required for RAG-powered content generation).

Status:

verification_pending

Step 2: Email Verification Request

Google Cloud Platform automatically sends a verification email to the domain owner. The email is sent to:

  • The email address associated with your domain's WHOIS record
  • Common administrative email addresses (admin@, webmaster@, etc.)
  • The email address registered with your domain registrar

Check Your Email

Look for an email from Google Cloud Platform with the subject line related to domain verification. This email typically arrives within a few minutes of datastore creation.

Step 3: Complete Verification

Follow the instructions in the verification email. This typically involves:

  1. Clicking a verification link in the email
  2. Confirming domain ownership in Google Search Console (if required)
  3. Waiting for Google to verify the domain (usually takes a few minutes to a few hours)

Step 4: Indexing Begins

Once verification is complete, Google automatically begins indexing your selected pages. The status in Antwork updates to reflect this progress.

Status progression:

verification_pending
→ Waiting for verification
indexing
→ Pages being indexed (24-48 hours)
active
→ Ready for AI-powered content generation

Technical Implementation

API Methods

Antwork uses Google Cloud Discovery Engine APIs to check verification status:

Primary Method (Basic Site Search)

fetchDomainVerificationStatus()

Fallback Method (Advanced Site Search)

listTargetSites()

Used for advanced site search datastores (required for RAG functionality)

Verification States

VERIFIED: Domain ownership confirmed
EXEMPTED: Public websites exempt from verification
UNVERIFIED: Verification pending or failed

Status Checking

Antwork automatically checks verification status:

  • When you visit the dashboard
  • Every 30 seconds while status is verification_pending
  • When you manually refresh the status

Troubleshooting

I didn't receive the verification email

  • Check your spam/junk folder
  • Verify the email address associated with your domain's WHOIS record
  • Check common administrative email addresses (admin@, webmaster@, etc.)
  • Wait a few hours - emails can sometimes be delayed
  • Contact your domain registrar to verify the registered email address

Verification is taking too long

Domain verification typically completes within:

  • Immediate to 1 hour: If you click the verification link promptly
  • 1-24 hours: If verification requires Google Search Console confirmation
  • Up to 48 hours: In rare cases with complex domain configurations

Status shows "verification_pending" but I verified

This can happen if:

  • The status check hasn't refreshed yet (check again in a few minutes)
  • You verified a different domain than the one configured
  • There's a delay in Google's status update system

Try clicking "Check Status" in the dashboard to manually refresh the verification status.

Expected Timeline

1

Datastore Creation

Immediate - happens when you add your website

2

Verification Email Sent

Within 5-10 minutes of datastore creation

3

Domain Verification

Immediate to 24 hours after clicking verification link

4

Page Indexing

24-48 hours after verification completes

Ready for Use

Once pages are indexed, your website content is available for AI-powered content generation

Need Help?

If you're experiencing issues with domain verification or have questions about the process, please contact our support team:

Contact Support

hello@antwork.io