Basic Plugin Development

Basic Plugin Development

This is an example of a Faraday Plugin that process a xml report.

Configure Custom Plugins Folder

To add custom plugins in faraday you first need to add the path where you have your plugins in the config.ini under server config section.


custom_plugins_folder = /home/user/.faraday/plugins

Place the plugin in the folder

The folder structure is: 

/home/user/.faraday/plugins/{new plugin name}/

inside the folder should be placed the .py plugin named as plugin.py
also there should be performed the command
  1. touch __init__.py


Plugin Code 

In this example we will create a plugin to analyze this XML

<?xml version="1.0" ?>
<!DOCTYPE example_tool>
<example_tool scanstart="Thu Nov  9 15:59:13 2017">
<details>
<item id="999979" ip="10.23.49.232" os="linux">
<uri>http://test.com/example.php</uri>
<issue severity="low">Some vuln text</issue>
</item>
<item id="39023023" ip="10.232.62.20" os="linux">
<uri>http://test.com/login.php</uri>
<issue severity="low">Some other text</issue>
</item>
<item id="8348343" ip="10.12.37.24" os="linux">
<uri>http://test.com/example.php</uri>
<issue severity="low">Yet another vuln text</issue>
</item>
<statistics elapsed="402" itemsfound="3" itemstested="10" />
</details>
</example_tool>

Plugin.py

from urllib.parse import urlparse
from faraday_plugins.plugins.plugin import PluginXMLFormat
import xml.etree.ElementTree as ET

class ExampleToolXmlParser:

    def __init__(self, xml_output):
        self.vulns = self.parse_xml(xml_output)

    def parse_xml(self, xml_output):
        vulns = []
        tree = ET.fromstring(xml_output)
        items = tree.iterfind('details/item')
        for item in items:
            ip = item.get('ip')
            os = item.get('os')
            uri = item.find('uri').text
            url = urlparse(uri)
            hostname = [url.netloc]
            path = url.path
            if url.scheme == 'https':
                port = 443
            else:
                port = 80
            issue = item.find('issue')
            severity = issue.get('severity')
            issue_text = issue.text
            vuln = {'ip': ip, 'uri': uri, 'os': os,
                    'hostname': hostname, 'port': port, 'path': path,
                    'issue_text': issue_text, 'severity': severity}
            vulns.append(vuln)
        return vulns


class ExampleToolPlugin(PluginXMLFormat):
    def __init__(self):
        super().__init__()
        self.identifier_tag = "example_tool"
        self.id = "example_tool"
        self.name = "Name of the tool"
        self.plugin_version = "0.0.1"

    def parseOutputString(self, output, debug=False):
        parser = ExampleToolXmlParser(output)
        for vuln in parser.vulns:
            h_id = self.createAndAddHost(vuln['ip'], vuln['os'], hostnames=vuln['hostname'])
            s_id = self.createAndAddServiceToHost(h_id, 'webserver', protocol='tcp', ports=vuln['port'])
            v_id = self.createAndAddVulnWebToService(h_id, s_id, vuln['issue_text'], severity=vuln['severity'],
                                                     path=vuln['path'])

def createPlugin():
    return ExampleToolPlugin()

The createPlugin function is the entry point, and it must return an instance of your plugin

Plugin Classes

PluginBase is the base class, right now we support Xml (PluginXMLFormat) and Json (PluginJsonFormat) files but you can add other by creating a subclass of PluginByExtension

In your plugin __init__ you must define the id, that will be the identifier of your plugin.

Beware that it must be unique, you can list all the plugins identifiers with this command: python -m faraday_plygins list


PluginByExtension:

This class adds the functionality off identify a report by its extension by setting the variable extension int the __init__

It can be one extension (".xml") o a list ("['.xml', '.xxx']") if your report can have multiple extensions


PluginXMLFormat:

Use this class if the plugin generates vulns from a xml file, with a .xml extension

If your report has xml format but a different extension (like nessus), remember to define the extension variable To identify the report set in the variable identifier_tag the main tag of the xml (example_tool in the example), it also can be one tag or a list of tags

class ExampleXMLTool(PluginXMLFormat):

    def __init__(self):
        super().__init__()
        self.identifier_tag = 'tool_tag'
        self.extension = ".xml"
        self.open_options = {"mode": "rb"} # Here you cant specify the mode to open the file, also the encoding ("encoding": "utf-8")

PluginJsonFormat:

Use this class if the plugin generates vulns from a json file, with a .json extension

If yor report has json format but a different extension, remember to define the extension variable To identify the report set in the variable json_keys a set with some of the identifiers keys of the json object

class ExampleJsonTool(PluginJsonFormat):

    def __init__(self):
        super().__init__()
        self.json_keys = {'name', 'hosts'}
        self.extension = ".json"

PluginBase:

This class give you all the main methods to create hosts, services, vulnerabilities, credentials, etc.

def createAndAddHost(self, name, os="unknown", hostnames=None, mac=None, scan_template="", site_name="",
                    site_importance="", risk_score="", fingerprints="", fingerprints_software=""):

def createAndAddServiceToHost(self, host_id, name, protocol="tcp?", ports=None, status="open", version="unknown",
                            description=""):

def createAndAddVulnToHost(self, host_id, name, desc="", ref=None, severity="", resolution="", vulnerable_since="", 
                            scan_id="", pci="", data="", external_id=None, run_date=None):

def createAndAddVulnToService(self, host_id, service_id, name, desc="", ref=None, severity="", resolution="", 
                            risk="", data="", external_id=None, run_date=None):

def createAndAddVulnWebToService(self, host_id, service_id, name, desc="", ref=None, severity="", resolution="",
                                website="", path="", request="", response="", method="", pname="",
                                params="", query="", category="", data="", external_id=None, run_date=None):
   
def createAndAddCredToService(self, host_id, service_id, username, password):
    

Test and Debug

You can test your plugin by enable the custom_plugins_folder setting, and try it with faraday

You can also test your plugin from the command line.

Commands

List all available plugins

Verify that the plugins is loaded by the plugin manager
python -m faraday_plugins list

Available Plugins:
...
...
...
example_tool - Name of the tool
Loaded Plugins: 64

Test plugin report detection

Verify that your file is detected by your plugin

python -m faraday_plugins detect /path/to/report.xml

Plugin: example_tool

Test plugin process report

Verify that your plugin parses the file ok and generate the json structure that will be loaded into faraday

python -m faraday_plugins process example_tool  /path/to/report.xml


{"hosts": [{"ip": "10.23.49.232", "os": "linux", "hostnames": ["test.com"], "description": "", "mac": null, "credentials": [], "services": [{"name": "webserver", "protocol": "tcp", "port": 80, "status": "open", "version": "unknown", "description": "", "credentials": [], "vulnerabilities": [{"name": "Some vuln text", "desc": "", "severity": "low", "refs": [], "external_id": null, "type": "VulnerabilityWeb", "resolution": "", "data": "", "website": "", "path": "/example.php", "request": "", "response": "", "method": "", "pname": "", "params": "", "query": "", "category": ""}]}], "vulnerabilities": [], "scan_template": "", "site_name": "", "site_importance": "", "risk_score": "", "fingerprints": "", "fingerprints_software": ""}, {"ip": "10.232.62.20", "os": "linux", "hostnames": ["test.com"], "description": "", "mac": null, "credentials": [], "services": [{"name": "webserver", "protocol": "tcp", "port": 80, "status": "open", "version": "unknown", "description": "", "credentials": [], "vulnerabilities": [{"name": "Some other text", "desc": "", "severity": "low", "refs": [], "external_id": null, "type": "VulnerabilityWeb", "resolution": "", "data": "", "website": "", "path": "/login.php", "request": "", "response": "", "method": "", "pname": "", "params": "", "query": "", "category": ""}]}], "vulnerabilities": [], "scan_template": "", "site_name": "", "site_importance": "", "risk_score": "", "fingerprints": "", "fingerprints_software": ""}, {"ip": "10.12.37.24", "os": "linux", "hostnames": ["test.com"], "description": "", "mac": null, "credentials": [], "services": [{"name": "webserver", "protocol": "tcp", "port": 80, "status": "open", "version": "unknown", "description": "", "credentials": [], "vulnerabilities": [{"name": "Yet another vuln text", "desc": "", "severity": "low", "refs": [], "external_id": null, "type": "VulnerabilityWeb", "resolution": "", "data": "", "website": "", "path": "/example.php", "request": "", "response": "", "method": "", "pname": "", "params": "", "query": "", "category": ""}]}], "vulnerabilities": [], "scan_template": "", "site_name": "", "site_importance": "", "risk_score": "", "fingerprints": "", "fingerprints_software": ""}], "command": {"tool": "example_tool", "command": "example_tool", "params": "/path/to/report.xml", "user": "faraday", "hostname": "", "start_date": "2020-04-01T18:43:34.552623", "duration": 2650, "import_source": "report"}}

If you dont have faraday-server installed or dont have the custom_plugins_folder setting, you can use the --custom_plugins_folder parameter with any if the commands (list, detect and process)

Example:

python -m faraday_plugins list --custom-plugins-folder /home/user/.faraday/plugins/

Debug

In the PluginBase there is a logger defined in self.logger that you can use.

If you need to debug for plugins with the command line set this variable:

export PLUGIN_DEBUG=1
python -m faraday_plugins process appscan /path/to/report.xml
2019-11-15 20:37:03,355 - faraday.faraday_plugins.plugins.manager - INFO [manager.py:113 - _load_plugins()]  Loading Native Plugins...
2019-11-15 20:37:03,465 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [acunetix]
2019-11-15 20:37:03,495 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [amap]
2019-11-15 20:37:03,549 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [appscan]
2019-11-15 20:37:03,580 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [arachni]
2019-11-15 20:37:03,613 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [arp_scan]
2019-11-15 20:37:03,684 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [beef]
2019-11-15 20:37:03,714 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [brutexss]
2019-11-15 20:37:03,917 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [burp]
2019-11-15 20:37:03,940 - faraday.faraday_plugins.plugins.manager - DEBUG [manager.py:123 - _load_plugins()]  Load Plugin [dig]
...

    Still looking for answers? You can try opening a ticket.
      • Related Articles

      • Plugin List

        The main purpose of Faraday is to re-use the available tools in the community to take advantage of them in a multiuser way. You break it. We keep track of the pieces! To maximize flexibility, Faraday Plugins run only on the Client. This means you can ...
      • Faraday Plugin

        Intro In order to manage, add, and list information stored in faraday, we created fplugin, a simple plugin that allows you to interact directly with our Python API from the command line. It gives Faraday powerful scripting features and allows you to ...
      • Burp Proxy Extender Plugin

        This plugin is a script developed in Java as an extender to the Burp Proxy API (Pro/Community). Installation with NGINX and SSL Enable (Auto-Signed Certificate) To enable Burp Plugin using NGINX and with SSL Enable you need the Name (e.g. server FQDN ...
      • Import CSV using faraday_csv Plugin

        With faraday_csv Plugin, you can upload data to Faraday by using CSV files.  Main header: The main headers for faraday_csv Plugin are target or ip. Both columns contain the same information (host's IP). Without any of them, Faraday won't recognize ...
      • A plugin added too much information to my database

        You can go to the Manage - Vulns tab in the Web GUI, filter the vulnerabilities by whichever parameter you'd like, select them all and then click on Delete to remove them from the database. If you know which tool brought the information you want to ...