Skip to content

new(falco): add json_container support and http_output authorization support #3591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+72 −2
Open

new(falco): add json_container support and http_output authorization support #3591

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions falco.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -577,6 +577,18 @@ priority: debug
# programs to process and consume the data. By default, this option is disabled.
json_output: false

# [Incubating] `json_container`
#
# When json_output is enabled, Falco will output alert messages and events as JSON. If
# the json_container is also specified (optional), the event will be placed in that parent
# schema as defined in `falco_formats::format_json_container`
# Supported json containers:
# * `splunk_hec`: a container allowing direct posting to a Splunk HEC endpoint
# * `slack_webhook`: a container allowing direct posting to a Slack webhook endpoint
# * `generic_event_encoded`: a container that double encodes the json object as example container
# * "" or omitted from config: the JSON object without any container is emitted (default behaviour)
json_container: ""

# [Stable] `json_include_output_property`
#
# When using JSON output in Falco, you have the option to include the "output"
Expand Down Expand Up @@ -782,6 +794,8 @@ http_output:
keep_alive: false
# Maximum consecutive timeouts of libcurl to ignore
max_consecutive_timeouts: 5
# if provided, adds the value to the Authorization header in the POST
authorization: ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move these changes to a separate small PR that may be merged more quickly.


# [Stable] `program_output`
#
Expand Down
33 changes: 32 additions & 1 deletion userspace/engine/formats.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,14 @@ falco_formats::falco_formats(std::shared_ptr<const falco_engine> engine,
bool json_include_tags_property,
bool json_include_message_property,
bool json_include_output_fields_property,
const std::string &json_container,
bool time_format_iso_8601):
m_falco_engine(engine),
m_json_include_output_property(json_include_output_property),
m_json_include_tags_property(json_include_tags_property),
m_json_include_message_property(json_include_message_property),
m_json_include_output_fields_property(json_include_output_fields_property),
m_json_container(json_container),
m_time_format_iso_8601(time_format_iso_8601) {}

falco_formats::~falco_formats() {}
Expand Down Expand Up @@ -157,7 +159,7 @@ std::string falco_formats::format_event(sinsp_evt *evt,
}
}

return event.dump();
return format_json_container(event, evttime);
}

// should never get here until we only have OF_NORMAL and OF_JSON
Expand Down Expand Up @@ -197,3 +199,32 @@ std::map<std::string, std::string> falco_formats::get_field_values(

return ret;
}

/// @brief Helper function for encoding a json event into a parent container, described by
/// configuration.
/// @param json the json object to encode
/// @param evttime the event time
/// @return returns a std::string encoded json object according to the configured json_container
std::string falco_formats::format_json_container(nlohmann::json &json, time_t evttime) const {
if(m_json_container == std::string("splunk_hec")) {
nlohmann::json omsg;
omsg["time"] = evttime;
omsg["host"] = json["hostname"];
omsg["source"] = "falco";
omsg["event"] = json;
return omsg.dump();
} else if(m_json_container == std::string("slack_webhook")) {
nlohmann::json omsg;
omsg["text"] = json.dump();
return omsg.dump();
} else if(m_json_container == std::string("generic_event_encoded")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that this is a highly custom format. Perhaps a better approach would be to allow users to supply a formal schema via the configuration, supporting only officially recognized schema formats as templates. I'm curious to hear the perspective of other Falco maintainers on this.

There seem to be multiple possibilities. For instance, if you check out the sinsp_evt_formatter in libs, it provides a more abstracted way of formatting / transforming outputs in the project. A different approach may be more suitable here — we can determine the best path forward after gathering a few viewpoints.

I believe this is the main discussion point for this feature.

nlohmann::json omsg;
omsg["event"] = json.dump();
omsg["host"] = json["hostname"];
omsg["time"] = evttime;
const std::map<std::string, std::string> &const_labels = {{"evt_type", "falco"}};
omsg["fields"] = const_labels;
return omsg.dump();
}
return json.dump();
}
3 changes: 3 additions & 0 deletions userspace/engine/formats.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ class falco_formats {
bool json_include_tags_property,
bool json_include_message_property,
bool json_include_output_fields_property,
const std::string &json_container,
bool time_format_iso_8601);
virtual ~falco_formats();

Expand All @@ -47,12 +48,14 @@ class falco_formats {
std::map<std::string, std::string> get_field_values(sinsp_evt *evt,
const std::string &source,
const std::string &format) const;
std::string format_json_container(nlohmann::json &json, time_t evttime) const;

protected:
std::shared_ptr<const falco_engine> m_falco_engine;
bool m_json_include_output_property;
bool m_json_include_tags_property;
bool m_json_include_message_property;
bool m_json_include_output_fields_property;
const std::string m_json_container;
bool m_time_format_iso_8601;
};
1 change: 1 addition & 0 deletions userspace/falco/app/actions/init_outputs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ falco::app::run_result falco::app::actions::init_outputs(falco::app::state& s) {
s.config->m_json_include_tags_property,
s.config->m_json_include_message_property,
s.config->m_json_include_output_fields_property,
s.config->m_json_container,
s.config->m_output_timeout,
s.config->m_buffered_outputs,
s.config->m_outputs_queue_capacity,
Expand Down
6 changes: 6 additions & 0 deletions userspace/falco/config_json_schema.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,9 @@ const char config_schema_string[] = LONG_STRING_CONST(
"json_output": {
"type": "boolean"
},
"json_container": {
"type": "string"
},
"json_include_output_property": {
"type": "boolean"
},
Expand Down Expand Up @@ -543,6 +546,9 @@ const char config_schema_string[] = LONG_STRING_CONST(
},
"max_consecutive_timeouts": {
"type": "integer"
},
"authorization": {
"type": "string"
}
},
"minProperties": 1,
Expand Down
5 changes: 5 additions & 0 deletions userspace/falco/configuration.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ falco_configuration::falco_configuration():
m_json_include_tags_property(true),
m_json_include_message_property(false),
m_json_include_output_fields_property(true),
m_json_container(""),
m_rule_matching(falco_common::rule_matching::FIRST),
m_watch_config_files(true),
m_buffered_outputs(false),
Expand Down Expand Up @@ -347,6 +348,7 @@ void falco_configuration::load_yaml(const std::string &config_name) {
m_config.get_scalar<bool>("json_include_message_property", false);
m_json_include_output_fields_property =
m_config.get_scalar<bool>("json_include_output_fields_property", true);
m_json_container = m_config.get_scalar<std::string>("json_container", "");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my perspective, in such scenarios, we typically support direct boolean configurations in the config, which later translate into direct boolean checks. Alternatively, string inputs could be converted into flags (typically preferred) — for instance, check out the metrics_flags gating, which offers more performant and maintainable gates compared to repeated string comparisons over time.

Adding validity checks on user input while loading the config may be beneficial for free-form string inputs.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good callout - I'll migrate this to a flags var so the runtime check is minimal for the default path and shouldn't impact performance

Copy link
Contributor

@incertum incertum Jun 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would wait until we have discussed the user input. Maybe some sort of enum could also be an option or maybe at the end one simple boolean variable is all that is needed to enable this new user input formatting ...


m_outputs.clear();
falco::outputs::config file_output;
Expand Down Expand Up @@ -461,6 +463,9 @@ void falco_configuration::load_yaml(const std::string &config_name) {
m_config.get_scalar<uint8_t>("http_output.max_consecutive_timeouts", 5);
http_output.options["max_consecutive_timeouts"] = std::to_string(max_consecutive_timeouts);

std::string authorization;
authorization = m_config.get_scalar<std::string>("http_output.authorization", "");
http_output.options["authorization"] = authorization;
m_outputs.push_back(http_output);
}

Expand Down
2 changes: 2 additions & 0 deletions userspace/falco/configuration.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ class falco_configuration {
bool m_json_include_tags_property;
bool m_json_include_message_property;
bool m_json_include_output_fields_property;
std::string m_json_container;

std::string m_log_level;
std::vector<falco::outputs::config> m_outputs;

Expand Down
4 changes: 3 additions & 1 deletion userspace/falco/falco_outputs.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ falco_outputs::falco_outputs(std::shared_ptr<falco_engine> engine,
bool json_include_tags_property,
bool json_include_message_property,
bool json_include_output_fields_property,
const std::string &json_container,
uint32_t timeout,
bool buffered,
size_t outputs_queue_capacity,
Expand All @@ -56,6 +57,7 @@ falco_outputs::falco_outputs(std::shared_ptr<falco_engine> engine,
json_include_tags_property,
json_include_message_property,
json_include_output_fields_property,
json_container,
time_format_iso_8601)),
m_buffered(buffered),
m_json_output(json_output),
Expand Down Expand Up @@ -201,7 +203,7 @@ void falco_outputs::handle_msg(uint64_t ts,
jmsg["hostname"] = m_hostname;
jmsg["source"] = s_internal_source;

cmsg.msg = jmsg.dump();
cmsg.msg = m_formats->format_json_container(jmsg, evttime);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One consideration is that, with these changes, even users who do not actively utilize this new feature may experience the new overhead of these checks for each event. It might be valuable to gather additional feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's double-check whether this formatting also applies to the internal metrics output rules or other internal messages. I recall it involving slightly separate message handling, but I might be mistaken.

} else {
std::string timestr;
bool first = true;
Expand Down
1 change: 1 addition & 0 deletions userspace/falco/falco_outputs.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ class falco_outputs {
bool json_include_tags_property,
bool json_include_message_property,
bool json_include_output_fields_property,
const std::string &json_container,
uint32_t timeout,
bool buffered,
size_t outputs_queue_capacity,
Expand Down
5 changes: 5 additions & 0 deletions userspace/falco/outputs_http.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@ bool falco::outputs::output_http::init(const config &oc,
} else {
m_http_headers = curl_slist_append(m_http_headers, "Content-Type: text/plain");
}
if(!m_oc.options["authorization"].empty()) {
m_http_headers =
curl_slist_append(m_http_headers,
("Authorization: " + m_oc.options["authorization"]).c_str());
}
res = curl_easy_setopt(m_curl, CURLOPT_HTTPHEADER, m_http_headers);

// if the URL is quoted the quotes should be removed to satisfy libcurl expected format
Expand Down