Beginner OSINT Automation

Dear Edge Reader,

Today I would like to show you a simple way of automating some simple OSINT tasks. First of all, if you are a true beginner in CTI (Cyber Threat Intelligence), let me first explain what OSINT is. And since you are reading this From The Edge, why not throw in a bit analogy with astronomy!

BRING YOUR OWN OSINT TELESCOPE

OSINT, or Open Source Intelligence, is akin to a digital detective's toolkit for the information age. Imagine the vast expanse of the internet as a sprawling, ever-expanding galaxy. In this digital cosmos, OSINT is like a sophisticated telescope, allowing analysts to zoom in on specific stars of information amidst the infinite dark space of data.

Each website, social media platform, and online database is a star, emitting its own light in the form of publicly available data. OSINT experts navigate this galaxy with the precision of astrophysicists, using specialized tools and methodologies to chart courses and identify patterns.

Just as astronomers use telescopes to observe the celestial bodies, OSINT analysts use advanced software and analytical techniques to sift through the vastness of online data. They're not just passive observers; they're like interstellar cartographers, mapping out the contours of the digital universe to reveal hidden connections, patterns, and insights.

In this context, publicly accessible information - from news websites, forums, social networks, and more - is the raw material, the cosmic dust and gas, from which these analysts forge their understanding. This process is critical for cybersecurity, especially in a world where threats can emerge from any corner of this digital universe. By harnessing the power of OSINT, analysts can anticipate, identify, and mitigate cyber threats, essentially navigating through the cyber cosmos with the confidence and knowledge of seasoned space explorers.

OOOOR, … (if you are unamused …): In cybersecurity, OSINT (Open Source Intelligence) refers to the collection and analysis of publicly available data from the internet, such as websites, social media, and forums, to identify potential security threats and vulnerabilities.

Basically it’s all about some sort of look-ups into different databases, to gain some knowledge about the subject you are looking up.

OSINT AUTOMATION 101: Fetch domain inteligence

Lets walkthrough an example of how to fetch and display information about domains using the Domain OSINT API from "domainsdb.info".

Step 1: Open Google Apps Script

• Open a new or existing Google Sheet.

• Click on "Extensions" in the menu.

• Select "Apps Script" from the dropdown.

Step 2: Write the Custom Function

• In the Apps Script editor, you'll see a blank script file.

• Copy and paste the following code into the script editor (I just realized I need to add support for Code Formatting on my website!):

function fetchDomainInfo(domain) { 
  var apiUrl = "https://api.domainsdb.info/v1/domains/search?domain=" + domain; 
  try { 
    var response = UrlFetchApp.fetch(apiUrl);
    var data = JSON.parse(response.getContentText()); 
    var results = []; 
    if (data && data.domains) { 
      data.domains.forEach(function(domainInfo) {
        results.push([domainInfo.domain, domainInfo.create_date, domainInfo.update_date, domainInfo.country, domainInfo.isDead]);
      });
    } return results.length > 0 ? results : "No data found"; 
  } catch (e) {
    return "Error fetching data: " + e.toString();
  }
}

• Click the disk icon or "File" > "Save" to save your script.

Step 3: Use the Function in Google Sheets

• Go back to your Google Sheet.

• In a cell, type:

=fetchDomainInfo("example.com")

• replacing "example.com" with the domain you want to query.

• Press Enter, and the function will run, displaying information about the domain.

Step 4: Understand the Output

• The function will return an array containing the domain name, creation date, last update date, country, and status (active or dead).

• If the API does not find the domain or if an error occurs, it will display "No data found" or an error message.

Step 5: Customize and Expand

• Feel free to modify the script to suit your specific needs.

• For instance, you can adjust the output format or add more data points from the API response.

Tips for Success

• Check API Limits: Be aware of any rate limits imposed by the "domainsdb.info" API to avoid interruptions.

• Error Handling: The script includes basic error handling. You can expand this based on your specific requirements.

Conclusion

Congratulations! You've just unlocked the gateway to a fascinating realm of digital exploration with your newfound skill in automating domain data retrieval using the Domain OSINT API.

This is just your first step into a wider universe of OSINT, where every piece of openly available data is a star waiting to be discovered in the vast cosmos of the internet.

You can expand your intelligence gathering to other OSINT Sources, here is something to get you started in exploring:

1. Social Media Platforms:

   • Twitter, Facebook, Instagram, and LinkedIn provide real-time data on public sentiment, trends, and individual profiles.

   • Tools like TweetDeck or Social Searcher can be used to monitor social media activity.

2. Search Engines:

   • Google, Bing, DuckDuckGo, and Yandex offer extensive search capabilities to find publicly available information.

   • Custom search engines like Google Programmable Search Engine can be tailored for specific OSINT needs.

3. Domain and IP Analysis Tools:

   • WHOIS databases provide information on domain registrations.

   • Tools like VirusTotal, Shodan, or Censys.io offer insights into domain and IP address histories, associated security risks, and connected devices.

4. Geospatial Intelligence Tools:

   • Google Earth or Wikimapia for satellite imagery and geographical data.

   • MarineTraffic or FlightRadar24 for real-time tracking of ships and aircraft.

5. Data Breach Repositories:

   • Have I Been Pwned, DeHashed, and Intelligence X provide information on data breaches and leaked credentials.

6. Government and Public Records:

   • Government databases, public records, and legal documents available online.

   • Websites like PublicData.com and government portals for specific countries.

7. News Aggregators and Journals:

   • Google News, Feedly, and academic databases for current events, research papers, and articles.

   • Tools like Wayback Machine for accessing historical versions of web pages.

8. Forums and Online Communities:

   • Reddit, Stack Exchange, and niche forums for insights into specific communities or topics.

9. Cybersecurity Databases:

   • Databases like MITRE ATT&CK, CVE databases for vulnerabilities, and threat intelligence platforms.

Be sure to verify if they offer API for you to use and if it is Free to use.

What’s next

In one of the future guides, we will look into how can we now share the intelligence we obtained, with other parties.

… but for now, Happy Data Hunting!