Technical Documentation
Introduction
The Web API Bridge receives Web API calls as HTTP requests, which are translated into SQL Stored Procedure calls that are then invoked on either a local or remote MySQL server.
The Web API Bridge determines the MySQL server to connect to by performing a DNS TXT entry lookup. For example, if the domain of the API server is 'api.webapibridge.org', then a TXT entry lookup is performed on 'db.api.webapibridge.org'.
api.webapibridge.org → db.api.webapibridge.org
The format of the text entry is the name of the database without any numeral version numbers, the '@' symbol, then a comma separated list of hostnames.
webapibridge@db1.webapibridge.org,db2.webapibridge.org,...
Apart from DNS entries, the only configuration required is that x509 certificates and keys used for authentication with remote MySQL servers must be stored within the 'SSL_BASE' directory specified. Additionally, any targeted MySQL databases must also implement the following stored function, which allows the Web API Bridge to discover what parameters exported stored procedures expect.
Note, as indicated by the function implementation below, only stored procedures with a security type of 'DEFINER' and that have a comment of 'EXPORT' can be called.
DROP FUNCTION RetrieveParametersFor;
DELIMITER //
CREATE FUNCTION RetrieveParametersFor
(
$database CHAR(64),
$name CHAR(99)
)
RETURNS blob
READS SQL DATA
BEGIN
DECLARE $ret BLOB;
SELECT
param_list
INTO
$ret
FROM mysql.proc
WHERE db = $database
AND name = $name
AND type = 'PROCEDURE'
AND security_type = 'DEFINER'
AND comment = 'EXPORT'
ORDER BY modified DESC LIMIT 1;
return $ret;
END
//
DELIMITER ;
Configuration
As, conceptually, the Web API Bridge is not an authentication or authorisation layer, the Web API Bridge connects to any MySQL databases using the credentials "public" and "public", and only requires the 'EXECUTE' priviledge. However, best practise is that any connections to remote MySQL databases should also be authorised and ecrypted using x509 certificates.
GRANT EXECUTE ON *.* TO 'public'@'<Web API Brige IP address>' IDENTIFIED BY 'public';
The Web API Bridge configuration file allows the user to override the username and password used, as well as the location that x509 certificates (and keys) are stored within.
configuration/conf.php
<?php
define( "DB_USERNAME", "public" );
define( "DB_PASSWORD", "public" );
define( "SSL_BASE", "/etc/mysql/ssl/" );
To enable experimental code paths for specific api domains, edit the following by adding API domains to the array and mapping them to TRUE.
define( "EXPERIMENTAL", array( "api.example.com" => FALSE ) );
Underneath the 'SSL_BASE' directory should be a directory for each database server that is connected to. The name of the directory should match the fully qualified domain name of the server. Within that directory should be a client certificate file, key file, and a a server certificate authority (CA) file. For example:
/etc/mysql/ssl/db.webapibridge.org /etc/mysql/ssl/db.webapibridge.org/client-cert.pem /etc/mysql/ssl/db.webapibridge.org/client-key.pem /etc/mysql/ssl/db.webapibridge.org/servera-ca.pem
Apache Configuration
For each API server domain, an Apache configuration is required to specify which domains are allowed to access the API server by setting the 'Access-Control-Allow-Origin'.
<VirtualHost *:80>
ServerName api.%DOMAIN%
ServerAdmin webmaster@%DOMAIN%
DocumentRoot %DOCUMENT_ROOT%/WebAPIBridge/latest/webapibridge/sbin
# Uncomment to use specific database
#
#SetEnv USE_DATABASE <database name>@<database hostname>
# Uncomment to enable EXPERIMENTAL code paths
#
#SetEnv USE_EXPERIMENTAL TRUE
#
# Uncomment to allow /api/auth/oauth2/token return mingled sessionid/csrf token
#
#SetEnv USE_OAUTH2_TOKEN TRUE
# Uncomment to not set server side sessionid cookie
#
#SetEnv USE_SESSION_COOKIE FALSE
# Uncomment to only log level 0 messages
#
#SetEnv SHOW_LOG_TO_LEVEL 0
#
# Uncomment to log request parameters
#
#SetEnv LOG_REQUEST_PARAMETERS TRUE
#
# Uncomment to log actual stored procedure calls and number of rows returned
#
#SetEnv LOG_STORED_PROCEDURE_CALL TRUE
#
# Uncomment to log error messages returned
#
#SetEnv LOG_ERRORS TRUE
#
# Uncomment to override 200 response code with appropriate application error codes
#
#SetEnv USE_HTTP_ERROR_CODES TRUE
#
# Uncomment to set read timeout in seconds
#
#SetEnv USE_MYSQLI_OPT_READ_TIMEOUT 60
#
# Uncomment to set read timeout in seconds
#
#SetEnv USE_CACHE_RESULTS_FOR_SECS 60
<Directory %DOCUMENT_ROOT%/WebAPIBridge/latest/webapibridge/sbin/>
php_value auto_prepend_file "%DOCUMENT_ROOT%/WebAPIBridge/latest/webapibridge/configuration/conf.php"
</Directory>
<Directory %DOCUMENT_ROOT%/WebAPIBridge/latest/webapibridge/sbin/>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule /* index.php
</Directory>
ErrorLog /var/log/apache2/api.%DOMAIN%.log
ErrorLogFormat "[%t] [pid %P] [XForwardFor %{X-Forwarded-For}i] [Client %a] %M"
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog /var/log/apache2/access.log combined
#ErrorDocument 404 /index.php
Header set Access-Control-Allow-Origin "https://%DOMAIN%"
Header set Access-Control-Allow-Methods "POST, GET, PUT, OPTIONS, PATCH, DELETE"
Header set Access-Control-Allow-Credentials "true"
Header set Access-Control-Allow-Headers "X-Session-ID, X-CSRF-Token"
Header set Strict-Transport-Security "max-age=31536000"
Header set Content-Security-Policy "default-src 'none'; upgrade-insecure-requests;"
</VirtualHost>
Google Pub Sub configuration
The Web API Bridge is able to log API request meta-information to a Google Pub Sub topic.
Implementation
The Web API Bridge is currently implemented in PHP and is intended for use with Apache2, although it can be invoked on the PHP command-line for testing purposes. If appropriately configured, the Web API Bridge is able to log to a Google Pub Sub topic. Currently, this is facilitated by using the Google client libraries which are located in the 'dep/vendor' directory.
<?php
//require __DIR__ . '/dep/vendor/autoload.php';
The main Web API Bridge also uses the following generic dependecies whose implementations are described in the appendices.
include_once( "CSV.php" );
include_once( "strings.php" );
include_once( "jstream.php" );
include_once( "log.php" );
include_once( "ssl.php" );
include_once( "mysql.php" );
include_once( "input.php" );
include_once( "experimental.php" );
The 'main' function is invoked everytime a Web API request is received. It carries out the following key steps:
- Deteremine environment and filter input
- If a tunnelling request, call the tunnelled endpoint and return its response
- Translate the request URI into a provisional stored procedure name
- Determine the domain name to use for DNS queries
- Retrieve target database information from DNS (or local cache)
- Determine the version of the database to query
- Map any passed HTTP parameters to stored procedure arguments
- Call the stored procedure
- If the stored procedure was an authentication request it stores the returned session id in a cookie
- A response object is contructed
- A response object is returned as either CSV of JSON
- Meta-information about the call is logged to a Google Pub Sub topic.
$args = isset( $argv ) ? $argv : array();
main( $args );
function main( $argv )
{
if ( getenv( "USE_SYSLOG_PROGRAMNAME" ) ) define( "USE_SYSLOG_PROGRAMNAME", getenv( "USE_SYSLOG_PROGRAMNAME" ) );
if ( getenv( "USE_SYSLOG_FACILITY" ) ) define( "USE_SYSLOG_FACILITY", getenv( "USE_SYSLOG_FACILITY" ) );
if ( defined( "USE_SYSLOG_PROGRAMNAME" ) && defined( "USE_SYSLOG_FACILITY" ) )
{
openlog( USE_SYSLOG_PROGRAMNAME, LOG_PID, NeoLog_DecodeFacility( USE_SYSLOG_FACILITY ) );
}
switch( $_SERVER['REQUEST_METHOD'] )
{
case "OPTIONS":
case "PUT":
case "DELETE":
NeoLog( "Ignoring unsupported method " . $_SERVER['REQUEST_METHOD'] . ":" . $_SERVER["REDIRECT_URL"] . "?" . urldecode( file_get_contents( 'php://input') ) );
break;
default:
$start = microtime( TRUE );
$delta = 0;
$use_buffered = ( "TRUE" == getenv( "USE_BUFFERED" ) );
if
(
!string_has_prefix( $_SERVER["REDIRECT_URL"], "/auth/" )
&&
!string_has_prefix( $_SERVER["REDIRECT_URL"], "/api/" )
&&
!string_has_prefix( $_SERVER["REDIRECT_URL"], "/raw/" )
)
{
NeoLog( "Ignoring request:" . $_SERVER["REDIRECT_URL"] . "?" . urldecode( file_get_contents( 'php://input') ) );
exit();
}
else
{
$method = sprintf( "%4s", $_SERVER['REQUEST_METHOD'] );
if ( "TRUE" == getenv( "LOG_REQUEST_PARAMETERS" ) )
{
$request = GetURIDecodedRequest();
if ( array_key_exists( "password", $_REQUEST ) )
{
$request["password"] = "[hidden]";
}
$encoded = http_build_query( $request );
NeoLog( "Starting new request, $method " . $_SERVER["REDIRECT_URL"] . "?" . $encoded );
}
else
{
NeoLog( "Starting new request, $method " . $_SERVER["REDIRECT_URL"] );
}
}
NeoLogIn();
$environment = DetermineEnvironment ( $argv );
$sp_name = TranslateSPName ( $environment["redirect_url"] );
CheckIfTunnelRequest ( $sp_name, $environment );
$db_domain = DetermineDatabaseLookupDomain( $environment["server_name"] );
$db_info = LookupDatabaseInfo ( $db_domain );
$db_version = LookupDatabaseVersion ( $db_info );
CheckIfDownloadRequest ( $db_info, $db_version, $sp_name, $environment );
$sp_call = MapRequestToSPCall ( $db_info, $db_version, $sp_name, $environment["request"] );
$ret = CallStoredProcedure ( $db_info, $db_version, $sp_name, $sp_call, $environment );
SetSessionIDCookie ( $sp_name, $ret );
$response = CreateResponse ( $environment, $ret, ($delta = ceil( (microtime( TRUE ) - $start) * 1000000 )) );
OutputResponse ( $sp_name, $response, $environment["request"], $use_buffered );
// LogToPubSub ( $environment, $db_info, $db_version, $sp_call, $response, $delta );
NeoLogOut();
if ( "TRUE" == getenv( "LOG_STORED_PROCEDURE_CALL" ) )
{
//$count = 0;
if ( is_a( $ret, 'Results' ) )
{
LogResult( $sp_name, $ret, "returned" );
}
else
if ( is_a( $ret, 'EmptyResults' ) )
{
LogResult( $sp_name, $ret, "returned" );
}
else
if ( is_a( $ret, 'ErrorResults' ) )
{
LogResult( $sp_name, $ret, "threw ERROR: " . $ret->errorMessage );
}
else
{
NeoLog( "Called: " . $sp_name . ": no return" );
}
}
NeoLog( "Finishing:" . $_SERVER["REDIRECT_URL"] . " $delta microseconds" );
}
closelog();
}
function GetURIDecodedRequest()
{
$get = array_slice( $_REQUEST, 0, count( $_REQUEST ) );
$payload = array();
if ( "GET" != $_SERVER["REQUEST_METHOD"] )
{
$input = urldecode( file_get_contents( 'php://input' ) );
parse_str( $input, $payload );
}
return array_merge( $get, $payload );
}
function LogResult( $sp_name, $ret, $verb )
{
$count = $ret->numRows();
$inc = 4000;
$len = strlen( $ret->sqlQuery );
$message = "Called: " . $ret->sqlQuery . ": $verb " . $count . " rows";
$priority = String_Contains( $verb, "ERROR" ) ? LOG_ERR : LOG_INFO;
if ( $len < $inc )
{
NeoLog( $message, $priority );
}
else
{
NeoLog( "Called: " . $sp_name . ": $verb " . $count . " rows", $priority );
for ( $i = 0; $i < $len; $i += $inc )
{
NeoLog( substr( $ret->sqlQuery, $i, $inc ), $priority );
}
}
}
The following sections describes the implementations of these called functions in detail.
Determine Environment
function DetermineEnvironment( $argv )
{
NeoLog( "Determining environment" );
NeoLogIn();
$environment = null;
if ( "cli" != php_sapi_name() )
{
NeoLog( "Determining envionrment from Apache" );
$environment = DetermineEnvironmentFromApache();
}
else
{
NeoLog( "Determining envionrment from command-line arguments" );
$environment = DetermineEnvironmentFromArguments( $argv );
}
if ( ("" == $environment["server_name"]) || ("" == $environment["redirect_url"]) )
{
NeoLog( "Invalid environment, no SERVER_NAME or REDIRECT_URL." );
NeoLog( "Commandline usage:" );
NeoLog( "php index.php --server-name api.example.com --redirect-url /api/users/example/ --request '{\"param1\":\"value\",...}'" );
exit;
}
if ( !defined( "DB_USERNAME" ) ) define( "DB_USERNAME", "public" );
if ( !defined( "DB_PASSWORD" ) ) define( "DB_PASSWORD", "public" );
if ( !defined( "SSL_BASE" ) ) define( "SSL_BASE", "/etc/mysql/ssl/" );
NeoLog( "Configuration: username: " . DB_USERNAME . "; password: " . DB_PASSWORD . "; ssl_base: " . SSL_BASE );
//
// Setup use of CSRF Token
//
if ( "" != array_get( $environment['request'], "wab_csrf_token" ) )
{
$environment['request']["sid"]
=
substr( array_get( $environment['request'], "sid" ), 0, 32 )
.
substr( array_get( $environment['request'], "wab_csrf_token" ), 0, 32 );
}
NeoLogOut();
return $environment;
}
function DetermineEnvironmentFromApache()
{
NeoLogIn();
$input = urldecode( file_get_contents( 'php://input' ) );
$request = GetURIDecodedRequest();
$server_name = "";
$redirect_url = "";
$requested = "";
if ( isset( $_SERVER ) && array_key_exists( "SERVER_NAME", $_SERVER ) )
{
$server_name = $_SERVER["SERVER_NAME"];
}
if ( isset( $_SERVER ) && array_key_exists( "REDIRECT_URL", $_SERVER ) )
{
$redirect_url = CanonicalisePath( $_SERVER["REDIRECT_URL"] );
}
if ( isset( $request ) )
{
NeoLog( "Request is set" );
$request = string_has_prefix( $redirect_url, "/raw/" ) ? $request : Input::FilterInput( $request, null );
}
if ( "{" == substr( $input, 0, 1 ) )
{
if ( NULL !== json_decode( $input ) )
{
NeoLog( "Added JSON to request" );
$request["json"] = $input;
}
}
NeoLog( "Checking headers for API Key" );
NeoLogIn();
foreach( apache_request_headers() as $header => $value )
{
$_value = Input::Filter( $value );
if ( "Accept" == $header && !array_key_exists( "accept", $request ) )
{
$request["accept"] = $_value;
NeoLog( "request['accept'] = $_value" );
}
else
if ( "Authorization" == $header && !array_key_exists( "authorization", $request ) )
{
$request["authorization"] = $_value;
NeoLog( "request['authorization'] = $_value" );
if ( !array_key_exists( "sid", $request ) )
{
$request["sid"] = trim( str_replace( 'Bearer', '', $_value ) );
NeoLog( "request['sid'] = $_value (Bearer)" );
}
}
else
if ( "X-Access-ID" == $header && !array_key_exists( "accessid", $request ) )
{
$request["accessid"] = $_value;
NeoLog( "request['accessid'] = $_value" );
}
else
if ( "X-Api-Key" == $header && !array_key_exists( "apikey", $request ) )
{
$request["apikey"] = $_value;
NeoLog( "request['apikey'] = $_value" );
}
else
if ( "X-Auth-Token" == $header && !array_key_exists( "auth_token", $request ) )
{
$request["auth_token"] = $_value;
NeoLog( "request['auth_token'] = $_value" );
}
else
if ( "X-CSRF-Token" == $header && !array_key_exists( "wab_csrf_token", $request ) )
{
$request["wab_csrf_token"] = $_value;
NeoLog( "request['wab_csrf_token'] = $_value" );
}
else
if ( "X-Request-ID" == $header && !array_key_exists( "requestid", $request ) )
{
$request["requestid"] = $_value;
NeoLog( "request['requestid'] = $_value" );
}
else
if ( "X-Session-ID" == $header && !array_key_exists( "sid", $request ) )
{
$request["sid"] = $_value;
NeoLog( "request['sid'] = $_value" );
}
else
{
NeoLog( "Header: $header: $_value" );
}
}
NeoLogOut();
NeoLogOut();
return array
(
"server_name" => $server_name,
"redirect_url" => $redirect_url,
"request" => $request
);
}
function CanonicalisePath( $constant )
{
if ( '.php' == substr( $constant, -4 ) )
{
$constant = dirname( $constant );
}
if ( '/' != substr( $constant, -1 ) )
{
$constant .= "/";
}
return $constant;
}
function DetermineEnvironmentFromArguments( $argv )
{
$server_name = "";
$redirect_url = "";
$request = "";
$next_is_server_name = 0;
$next_is_redirect_url = 0;
$next_is_request = 0;
foreach( $argv as $arg )
{
if ( $next_is_server_name )
{
$server_name = $arg;
$next_is_server_name = 0;
}
else
if ( $next_is_redirect_url )
{
$redirect_url = $arg;
$next_is_redirect_url = 0;
}
else
if ( $next_is_request )
{
$request = json_decode( $arg, TRUE );
$next_is_request = 0;
}
else
{
switch ( $arg )
{
case "--server-name":
$next_is_server_name = 1;
break;
case "--redirect-url":
$next_is_redirect_url = 1;
break;
case "--request":
$next_is_request = 1;
break;
}
}
}
if ( !$request ) $request = array();
return array
(
"server_name" => $server_name,
"redirect_url" => $redirect_url,
"request" => $request
);
}
Translate stored procedure name
The stored procedure to be called is a simple mapping from the URL.
function TranslateSPName( $redirect_url )
{
NeoLog( "Translating stored procedure name from url" );
$bits = explode( '/', $redirect_url );
if ( "" == $bits[0] ) $bits = array_slice( $bits, 1 );
switch ( $bits[0] )
{
case 'api':
$sp_name = trim( implode( '_', array_slice( $bits, 1 ) ), '_' );
break;
case 'auth':
$sp_name = trim( implode( '_', array_slice( $bits, 0 ) ), '_' );
break;
case 'download':
$sp_name = 'download';
break;
case 'raw':
$sp_name = trim( implode( '_', array_slice( $bits, 0 ) ), '_' );
break;
case 'tunnel':
$sp_name = 'tunnel';
break;
default:
http_response_code( 402 );
//header( "Content-Type: text/plain" );
//echo "This is a Web API Bridge HTTP endpoint -- see https://www.webapibridge.org";
exit;
}
return $sp_name;
}
Check if tunnel request
function CheckIfTunnelRequest( $sp_name, $environment )
{
if ( "tunnel" != $sp_name )
{
//
// This is mysteriously not being executed...
//
NeoLog( "Checking if tunnel request - NO" );
}
else
{
NeoLog( "Checking if tunnel request - YES" );
NeoLogIn();
Tunnel( $environment );
NeoLogOut();
}
}
function Tunnel( $environment )
{
$request = $environment['request'];
$method = array_get( $request, "wab_method" );
$authorization = array_get( $request, "wab_authorization" );
$host = array_get( $request, "wab_host" );
$endpoint = array_get( $request, "wab_endpoint" );
$accept = array_get( $request, "wab_accept" );
$content_type = array_get( $request, "wab_content_type" );
$content64 = array_get( $request, "wab_content64" );
$headers = array_get( $request, "wab_headers" );
NeoLog( "WAB tunnel debug: $host$endpoint" );
if ( ! ($method && $host && $endpoint && $accept && $content_type) )
{
header( "Content-Type: text/plain" );
echo "Error: incomplete headers" . "\n";
echo "+ wab_method: $method" . "\n";
echo "+ wab_host: $host" . "\n";
echo "+ wab_endpoint: $endpoint" . "\n";
echo "+ wab_accept: $accept" . "\n";
echo "+ wab_content_type: $content_type" . "\n";
echo "+ wab_content64: $content64" . "\n";
}
else
{
$curl = SetupCURL( $method, $authorization, $host, $endpoint, $content_type, $content64, $accept, $headers );
$response = curl_exec ( $curl );
$err = curl_error( $curl );
if ( $err )
{
header( "Content-Type: text/plain" );
echo $err;
}
else
{
header( "Content-Type: $accept" );
echo $response;
}
}
ob_flush();
flush();
exit;
}
function SetupCURL( $method, $authorization, $host, $endpoint, $content_type, $content64, $accept, $headers )
{
$curl = curl_init();
$verify_host = 0;
$verify_peer = 0;
$verify = ! string_has_suffix( $host, ".local" );
if ( $verify )
{
$verify_host = 1;
$verify_peer = 1;
}
$headers = array
(
"Accept: $accept",
"Content-Type: $content_type",
"Cache-Control: no-cache"
);
if ( "" != $authorization )
{
$headers[] = "Authorization: $authorization";
}
$url = $host . $endpoint;
$content = base64_decode( $content64 );
if ( "GET" == $method )
{
$url = $url . "?" . $content;
$content = "";
}
else
{
$payload = array();
$parameters = explode( "&", $content );
foreach ( $parameters as $keyval )
{
$bits = explode( "=", $keyval );
$key = $bits[0];
$val = $bits[1];
$payload[$key] = $val;
}
$content = http_build_query( $payload );
}
error_log( "WAB Tunnel: $url" );
curl_setopt_array(
$curl,
array
(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => $method,
CURLOPT_POSTFIELDS => $content,
CURLOPT_HTTPHEADER => $headers,
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0
)
);
return $curl;
}
static function ExtractParameters( $request )
{
$parameters = array();
foreach( $request as $key => $value )
{
if ( "wab_" != substr( $key, 0, 4 ) )
{
$parameters[$key] = $value;
}
}
return $parameters;
}
Determine database domain
In order to allow a Web API Bridge to act as an API service for arbitrary domains, the hostname (or IP address) of the database server to be connected to, as well as the name of the database instance to be interrogated, is configured using DNS text records.
The prefix 'db.' is added to the API domain called to create the domain that is queried -- for example, 'api.example.com' would become 'db.api.example.com'. If a local domain is called, such as 'api.example.com.local', the prefix 'local-db.' is added to the API domain called, and the '.local' suffix is removed, to create the domain that is queried -- for example, 'api.example.com.local' would become 'local-db.api.example.com'.
function DetermineDatabaseLookupDomain( $server_name )
{
if ( string_has_suffix( $server_name, ".local" ) )
{
$db_domain = "local-db." . str_replace( '.local', '', $server_name );
}
else
{
$db_domain = "db." . $server_name;
}
NeoLog( "Determining database lookup domain name: $db_domain" );
return $db_domain;
}
Lookup database info
function LookupDatabaseInfo( $db_domain )
{
$db_info = null;
$db_temp = (object) array( "dbname" => null, "hosts" => null );
if ( ($database = getenv( "USE_DATABASE" )) )
{
$db_info = ExtractDBInfoFromEnvText( $database );
}
else
{
if ( function_exists( "apcu_enabled" ) && apcu_enabled() && apcu_exists( $db_domain ) )
{
$db_cached = apcu_fetch( $db_domain );
$db_name = $db_cached->dbname;
$db_host = $db_cached->hosts[0];
if ( $db_name && $db_host )
{
NeoLog( "Looking up database info: found cached: $db_name@$db_host" );
$db_info = $db_cached;
}
}
if ( ! $db_info )
{
try
{
$records = dns_get_record( $db_domain, DNS_TXT );
$n = (null != $records) ? count( $records ) : 0;
if ( 0 < $n )
{
$db_dns = ExtractDBInfoFromDNSRecords( $records );
}
else
{
$db_dns = ExtractDBInfoFromDBDomain( $db_domain );
}
$db_name = $db_dns->dbname;
$db_host = $db_dns->hosts[0];
if ( $db_name && $db_host )
{
NeoLog( "Looking up database info: DNS returned $n records: $db_name@$db_host" );
$db_info = $db_dns;
}
if ( $db_info && function_exists( "apcu_enabled" ) && apcu_enabled() )
{
$ttl = 60; // seconds
apcu_add( $db_domain, $db_info, $ttl );
NeoLogIn();
NeoLog( "Caching: added to cache: $db_domain" );
NeoLogOut();
}
}
catch ( Exception $ex )
{
NeoLog( "Exception while retrieving from DNS" );
exit;
}
}
}
NeoLogIn();
if ( ! $db_info )
{
NeoLogError( "Error, Could not determine database information, exiting." );
exit;
}
else
{
// Following will abort if cannot retrieve database connection.
CheckDatabaseConnection( $db_info );
}
NeoLogOut();
return $db_info;
}
function ExtractDBInfoFromDNSRecords( $records )
{
foreach ( $records as $record )
{
$host = array_key_exists( "host", $record ) ? $record[ "host"] : "";
$class = array_key_exists( "class", $record ) ? $record[ "class"] : "";
$ttl = array_key_exists( "ttl", $record ) ? $record[ "ttl"] : "";
$type = array_key_exists( "type", $record ) ? $record[ "type"] : "";
$txt = array_key_exists( "txt", $record ) ? $record[ "txt"] : "";
if ( $txt )
{
return ExtractDBInfoFromDNSText( $txt );
}
}
return null;
}
function ExtractDBInfoFromDNSText( $txt )
{
$bits = explode( "@", $txt );
switch ( count( $bits ) )
{
case 2:
$dbname = $bits[0];
$hosts = explode( ",", $bits[1] );
break;
case 1:
$hosts = explode( ",", $bits[0] );
break;
}
return (object) array( "dbname" => $dbname, "hosts" => $hosts );
}
/*
* Important!!! Below code is different to similar code above.
* This function is called when the USE_DATABASE environment variable is used.
* This may either be: <database>, e.g., base
* or <database>@[host1],..., e.g., base@db1.example.com,db2.example.com
*
* In contrast, code above allows a DNS text record to have only hostnames.
*/
function ExtractDBInfoFromEnvText( $txt )
{
$bits = explode( "@", $txt );
switch ( count( $bits ) )
{
case 2:
$dbname = $bits[0];
$hosts = explode( ",", $bits[1] );
break;
case 1:
$dbname = $bits[0];
$hosts = array( "localhost" );
break;
}
return (object) array( "dbname" => $dbname, "hosts" => $hosts );
}
function ExtractDBInfoFromDBDomain( $db_domain )
{
$truncated = str_replace( "www.", "", $db_domain );
$truncated = str_replace( "ro-api-", "", $truncated );
$truncated = str_replace( "ro-api.", "", $truncated );
$truncated = str_replace( "api-", "", $truncated );
$truncated = str_replace( "api.", "", $truncated );
$bits = explode( ".", $truncated );
$n = count( $bits );
for ( $i = 1; $i < $n; $i++ )
{
if ( ! IsTLD( $bits[$i], $i ) )
{
$dbname = $bits[$i];
break;
}
}
return (object) array( "dbname" => $dbname, "hosts" => array("localhost") );
}
function IsTLD( $domain_part, $level = 1 )
{
switch ( $level )
{
case "1":
switch ( $domain_part )
{
case "au":
case "com":
case "net";
case "org";
case "co":
case "info":
case "xyz":
case "local":
return true;
default:
return false;
}
break;
case "2":
switch ( $domain_part )
{
case "com":
case "net";
case "org";
case "id":
case "conf":
return true;
default:
return false;
}
break;
default:
return false;
}
}
function CheckDatabaseConnection( $db_info )
{
if ( !defined( "DB_HOSTNAME" ) ) define( "DB_HOSTNAME", $db_info->hosts[0] );
NeoLog( "Checking database connection to: " . $db_info->hosts[0] );
NeoLogIn();
$m = MySQLCreate();
NeoLogOut();
return TRUE;
}
Determine database version
function LookupDatabaseVersion( $db_info )
{
$db_version = "";
$db_name = "";
$db_hosts = implode( ', ', $db_info->hosts );
$db_host = $db_info->hosts[0];
$mysql = MySQLCreate();
$tuples = MySQLInfo( $mysql, "SHOW DATABASES" );
foreach( $tuples as $tuple )
{
$tmp = $tuple["Database"];
if ( string_has_prefix( $tmp, $db_info->dbname ) )
{
$db_name = $tmp;
}
}
$db_version = $db_name ? str_replace( $db_info->dbname, "", $db_name ) : $db_version;
if ( $db_name )
{
NeoLog( "Determining target database version: found: $db_name" );
}
else
{
NeoLogError( "Determining target database version: cound not find database for: " . $db_info->dbname );
exit;
}
return $db_version;
}
Check if download request
function CheckIfDownloadRequest( $db_info, $db_version, $sp_name, $environment )
{
NeoLog( "Checking if tunnel request" );
if ( "download" != $sp_name )
{
//
// This is mysteriously not being executed...
//
NeoLog( "Checking if download request - NO" );
}
else
{
NeoLog( "Performing download" );
NeoLogIn();
Download( $db_info, $db_version, $environment );
NeoLogOut();
}
}
function Download( $db_info, $db_version, $environment )
{
$request = $environment['request'];
$db = $db_info->dbname . $db_version;
$sid = array_get( $request, "sid" );
$token = array_get( $request, "token" );
if ( $token )
{
$file = Retrieve( $db, $sid, $token );
if ( $file )
{
$name = $file->filename;
$type = $file->filetype;
$size = $file->filesize;
$base64 = $file->base64;
$content = base64_decode( $base64 );
header( "Content-Type: application/octet-stream" );
header( "Content-Length: $size" );
header( "Content-Disposition: attachment; filename=\"$name\"" );
header( "Pragma: public" );
header( "Cache-Control: must-revalidate, post-check=0, pre-check=0" );
echo $content;
}
else
{
header( "Content-Type: text/plain" );
echo "File not found";
}
}
else
{
header( "Content-Type: text/plain" );
echo "No file specified";
}
ob_flush();
flush();
exit;
}
function Retrieve( $db, $sid, $token )
{
$sp_call = (object) array
(
"proc_name" => "Base_Files_Retrieve_By_Token",
"arguments" => array( $sid, $token )
);
$ret = MySQLProcedure( $db, $sp_call, $procedure );
//NeoLog( "Retrieve: CALL $procedure" );
return array_key_exists( 0, $ret ) ? $ret[0] : null;
}
Determine Stored Procedure Call
function MapRequestToSPCall( $db_info, $db_version, $sp_name, $request )
{
$sp_call = null;
$proc_name = null;
$arguments = array();
if ( "multiselect" != $sp_name )
{
NeoLog( "Determine Stored Procedure Call" );
NeoLogIn();
$db = $db_info->dbname . $db_version;
//
// Create list of provisional sp names.
//
$provisional[] = $sp_name;
if ( string_has_suffix( $sp_name, "_csv" ) )
{
$provisional[] = preg_replace( "/_csv\z/", "", $sp_name );
}
$provisional[] = $provisional[count($provisional)-1] . "_retrieve";
//
// Use first procedure found that matches provisional name.
//
$result = null;
foreach( $provisional as $prov )
{
$sql = "RetrieveParametersFor( '$db', '$prov' )";
$result = MySQLFunction( $db, $sql );
if ( null !== $result )
{
$proc_name = $prov;
break;
}
}
if ( !$proc_name )
{
NeoLog( "Could not find procedure for: " . $sp_name );
return null;
}
else
{
$parameters = ConvertParametersToDictionary( $result );
foreach ( $parameters as $parameter )
{
// Strip of leading '$' and make lowercase.
$parameter_name = str_replace( '$', '', $parameter );
$value = null;
if ( "wab_remote_address" == $parameter_name )
{
$value = $_SERVER["REMOTE_ADDR"];
}
else
if ( "sid" == strtolower( $parameter_name ) )
{
$sid = array_get( $request, "sid" );
$remote_addr = GetRemoteAddress();
if ( Experimental() )
{
$value = $sid . "@" . $remote_addr;
}
else
{
$value = $sid;
}
}
else
if ( "apikey" == strtolower( $parameter_name ) )
{
$value = array_get( $request, "apikey" ) . "@" . $_SERVER["REMOTE_ADDR"];
}
else
if ( array_key_exists( $parameter_name, $request ) )
{
$value = array_get( $request, $parameter_name );
}
else
if ( array_key_exists( strtolower( $parameter_name ), $request ) )
{
$value = array_get( $request, strtolower( $parameter_name ) );
}
else
{
if ( ("PATCH" == $_SERVER["REQUEST_METHOD"]) )
{
$value = null;
}
else
{
$value = "";
}
}
$arguments[] = $value;
}
}
NeoLogOut();
}
return (object) array( "proc_name" => $proc_name, "arguments" => $arguments );
}
Get Remote Address
function GetRemoteAddress()
{
if ( array_key_exists( "X-Forwarded-For", $_SERVER ) && $_SERVER["X-Forwarded-For"] )
{
return strtok( $_SERVER["X-Forwarded-For"], "," );
}
else
{
return $_SERVER["REMOTE_ADDR"];
}
}
Convert parameters to dictionary
function ConvertParametersToDictionary( $string )
{
$parameters = array();
$bits = explode( ",", trim( $string ) );
foreach ( $bits as $parameter )
{
$parts = explode( " ", trim( $parameter ) );
if ( '_' != $parts[0][0] )
{
$parameters[] = $parts[0];
if ( defined( "VERBOSE" ) && VERBOSE ) error_log( $parts[0] . " " . $parts[1] );
}
}
return $parameters;
}
Call stored procedure
function CallStoredProcedure( $db_info, $db_version, $sp_name, $sp_call, $environment )
{
$ret = null;
if ( $sp_call )
{
NeoLog( "Call Stored Procedure" );
NeoLogIn();
$db = $db_info->dbname . $db_version;
$cache_seconds = intval( getenv( "USE_CACHE_RESULTS_FOR_SECS" ) );
if ( "multiselect" == $sp_name )
{
$ret = Multiselect ( $db_info, $db_version, $environment );
}
else
if ( $cache_seconds && function_exists( "apcu_enabled" ) && apcu_enabled() )
{
$key = $sp_call->proc_name . "(" . join( ",", $sp_call->arguments ) . ")";
NeoLog( "Key: $key" );
if ( apcu_exists( $key ) )
{
error_log( "Returning cached error value (cached for $cache_seconds seconds)" );
$ret = apcu_fetch( $key );
}
else
{
$ret = MySQLProcedureResults( $db, $sp_call );
if ( $ret instanceof ErrorResults )
{
apcu_store( $key, $ret, $cache_seconds );
}
}
}
else
{
$ret = MySQLProcedureResults( $db, $sp_call );
}
NeoLogOut();
}
return $ret;
}
function Multiselect( $db_info, $db_version, $environment )
{
$use_new = true;
$set = array();
$db = $db_info->dbname . $db_version;
$kinds = explode( ',', array_get( $environment['request'], "kinds" ) );
$sid = array_get( $environment['request'], "sid" );
$filter = array_get( $environment['request'], "filter" );
#
# Create pseudo-request for select call
#
$entity_encoded = array_get( $environment['request'], "json" );
$json = html_entity_decode( $entity_encoded );
$request = (array) json_decode( $json );
$request['filter'] = $filter;
$request['sid'] = array_get( $environment['request'], "sid" );
foreach ( $kinds as $kind )
{
$sp_call = null;
$bits = explode( ":", $kind );
switch ( count( $bits ) )
{
case 1:
$id = "";
$select_name = $bits[0];
break;
case 2:
$id = explode( ':', $kind )[0];
$select_name = explode( ':', $kind )[1];
break;
default:
break;
}
if ( $select_name )
{
$proc_name = "selects_$select_name";
if ( $use_new )
{
$request['id'] = $id;
$sp_call = MapRequestToSPCall( $db_info, $db_version, $proc_name, $request );
$tuples = MySQLProcedure( $db, $sp_call, $procedure );
}
else
{
$arguments = array();
$arguments[] = $sid;
$arguments[] = $id;
$arguments[] = "";
$arguments[] = $filter;
$sp_call = (object) array
(
"proc_name" => $proc_name,
"arguments" => $arguments
);
$tuples = MySQLProcedure( $db, $sp_call, $procedure );
}
if ( is_array( $tuples ) )
{
$set[] = (object) array( "id" => $id, "name" => $kind, "tuples" => $tuples, "call" => $procedure );
}
else
{
$set[] = (object) array( "id" => $id, "name" => $kind, "error" => $tuples, "call" => $procedure );
}
}
}
return $set;
}
Set session ID cookie
function SetSessionIDCookie( $sp_name, $ret )
{
NeoLog( "Existing sessionid: " . array_get( $_COOKIE, "sid" ) );
NeoLog( "USE_SESSION_COOKIE: " . getenv( "USE_SESSION_COOKIE" ) );
if ( "FALSE" != getenv( "USE_SESSION_COOKIE" ) )
{
$is_local = ("8443" == $_SERVER['SERVER_PORT']);
if ( !$is_local ) $is_local = string_has_suffix( $_SERVER['SERVER_NAME'], '.test' );
if ( "auth_login" == $sp_name )
{
if ( is_a( $ret, 'Results' ) )
{
$tuple = $ret->getFirst(); // Retrieve first tuple.
$sessionid = $tuple->sessionid;
$cookie = "Set-Cookie: sid=";
$cookie = $cookie . $sessionid;
if ( $is_local )
{
// SameSite is causing issues with local API server
// that uses self-signed certificates.
$cookie = $cookie . "; path=/; HttpOnly;secure";
}
else
{
$cookie = $cookie . "; path=/; HttpOnly;secure;SameSite=strict";
}
header( $cookie );
NeoLog( "Setting Session ID Cookie: " . $sessionid );
}
}
else
if ( "auth_logout" == $sp_name )
{
header( "Set-Cookie: sid=deleted; path=/; HttpOnly;secure;SameSite=strict; expires=Thu, 01 Jan 1970 00:00:00 GMT" );
NeoLog( "Clearing Session ID Cookie" );
}
}
if ( "TRUE" == getenv( "USE_OAUTH2_TOKEN" ) )
{
if ( "auth_oauth2_token" == $sp_name )
{
if ( is_a( $ret, 'Results' ) )
{
$tuple = $ret->getFirst(); // Retrieve first tuple.
if ( $tuple && $tuple->access_token )
{
$access_token = $tuple->access_token;
echo '{' . '"access_token"' . ':' . '"' . $access_token . '"' . '}';
NeoLog( "Returning OAuth Token: " . $access_token );
exit( 0 );
}
}
}
}
}
CreateResponse
The response from the Web API Bridge is a hierarchical structure encoded in JSON. The outermost object contains the following members that can be used for debugging.
function CreateResponse( $environment, $ret, $microseconds)
{
$request = $environment["request"];
$response = array();
$response['URL' ] = $environment["redirect_url"];
$response['error' ] = is_array( $ret ) || is_a( $ret, 'Results' ) || is_a( $ret, 'EmptyResults' ) ? "" : ( is_a( $ret, 'ErrorResults' ) ? $ret->errorMessage : $ret );
$response['failover' ] = "FALSE";
$response['hostname' ] = "";
$response['message' ] = "";
$response['status' ] = is_array( $ret ) || is_a( $ret, 'Results' ) || is_a( $ret, 'EmptyResults' ) ? "OK" : "ERROR";
$response['warning' ] = "";
$response['target_id'] = array_get( $request, "target_id" );
$response['submit' ] = array_get( $request, "submit" );
$response['limit' ] = array_get( $request, "limit" );
$response['offset' ] = array_get( $request, "offset" );
$response['microsecs'] = $microseconds;
$response['results' ] = is_array( $ret ) || is_a( $ret, 'Results' ) ? $ret : (is_a( $ret, 'EmptyResults' ) ? array() : null);
if ( "TRUE" == getenv( "USE_HTTP_ERROR_CODES" ) )
{
if ( string_contains( $response["status"], "ERROR" ) )
{
if ( string_contains( $response["error"], "INVALID_APIKEY" ) )
{
http_response_code( 403 );
}
else
if ( string_contains( $response["error"], "INVALID_AUTHORISATION" ) )
{
http_response_code( 403 );
}
else
if ( is_null( $ret ) )
{
http_response_code( 501 );
}
else
{
http_response_code( 400 );
}
}
}
if ( string_contains( $response["error"], "DUPLICATE_REQUEST" ) )
{
$response["status"] = "DUPLICATE";
}
if ( !array_key_exists( 'accept', $request ) ) $request['accept'] = "";
if ( is_null( $ret ) )
{
$response['error'] = "Could not resolve stored procedure.";
}
if ( "TRUE" == getenv( "LOG_ERRORS" ) )
{
if ( $response['error'] )
{
NeoLogError( "Error: " . $response['error'] );
}
}
return (object) $response;
}
Output Response
function OutputResponse( $sp_name, $response, $request, $use_buffered = false )
{
NeoLog( "Output Response" );
NeoLogIn();
if ( string_has_suffix( $sp_name, "_csv" ) )
{
TranslateToCSV ( $response, $request );
}
else
if ( $use_buffered )
{
TranslateToJSON( $response );
}
else
{
TranslateToJSONUnbuffered( $response );
}
NeoLogOut();
}
Translate to JSON
Finally works! Refer to: https:stackoverflow.com/questions/15273570/continue-processing-php-after-sending-http-response
Especially, Brian's comment regarding the need for "Content-Encoding" to be set to "none".
function TranslateToJSON( $response )
{
NeoLog( "Translate to JSON (Buffered)" );
if ( ob_get_contents() ) ob_end_clean();
set_time_limit(0);
ignore_user_abort( TRUE );
ob_start();
{
\JStream::Encode( 0, $response );
echo "\n";
$size = ob_get_length();
header( "Content-Length: $size" );
header( "Connection: close" );
header( "Content-Encoding: none" );
header( "Content-Type: application/json" );
}
ob_end_flush();
ob_flush();
flush();
if (session_id()) session_write_close();
}
function TranslateToJSONUnbuffered( $response )
{
NeoLog( "Translate to JSON (Unbuffered)" );
set_time_limit(0);
ignore_user_abort( TRUE );
{
header( "Connection: close" );
header( "Content-Encoding: none" );
header( "Content-Type: application/json" );
\JStream::Encode( 0, $response );
echo "\n";
}
flush();
if (session_id()) session_write_close();
}
function TranslateToCSV( $response, $request )
{
NeoLog( "Translate to CSV" );
if ( "OK" == $response->status )
{
$content = "";
$ds = ("true" == array_get( $request, "double_space" ));
$numbered = ("true" == array_get( $request, "numbered" ));
$map = GenerateMap( array_get( $request, "csv_map" ) );
$results = $response->results;
header( "Content-Type: application/octet-stream" );
header( "Content-Disposition: attachment;filename=\"download.csv\"" );
header( "Pragma: public" );
header( "Cache-Control: must-revalidate, post-check=0, pre-check=0" );
header( "Connection: close" );
if ( $map )
{
echo \CSV::encode( \CSV::MapValues( $map, $results->toArray() ), $numbered, $ds );
}
else
{
\CSV::Echo( $results, $numbered, $ds );
}
flush();
}
}
function TranslateToCSVOrig( $response, $request )
{
NeoLog( "Translate to CSV" );
NeoLog( var_export( $response, true ) );
if ( "OK" == $response->status )
{
$content = "";
$ds = ("true" == array_get( $request, "double_space" ));
$numbered = ("true" == array_get( $request, "numbered" ));
$map = GenerateMap( array_get( $request, "csv_map" ) );
$results = $response->results->toArray();
if ( $map )
{
NeoLog( "Calling CSV::encode with mapped tuples." );
$content = \CSV::encode( \CSV::MapValues( $map, $results ), $numbered, $ds );
}
else
{
NeoLog( "Calling CSV::encode with unmapped tuples." );
$content = \CSV::encode( $results, $numbered, $ds );
}
$size = strlen( $content );
if ( false )
{
error_log( "Content-Type: application/octet-stream" );
error_log( "Content-Length: $size" );
error_log( "Content-Disposition: attachment;filename=\"download\"" );
}
header( "Content-Type: application/octet-stream" );
header( "Content-Length: $size" );
header( "Content-Disposition: attachment;filename=\"download.csv\"" );
header( "Pragma: public" );
header( "Cache-Control: must-revalidate, post-check=0, pre-check=0" );
header( "Connection: close" );
echo $content;
flush();
exit;
}
}
function GenerateMap( $base64_encoded_csv_map )
{
$csv_map = base64_decode( $base64_encoded_csv_map );
if ( defined( "VERBOSE" ) && VERBOSE ) error_log( "GenerateMap( \"$csv_map\" )" );
//
// csv_map=Family+Name|family_name,...
//
$map = null;
if ( $csv_map )
{
$map = array();
$bits = explode( ",", $csv_map );
foreach ( $bits as $mapping )
{
$pieces = explode( "|", $mapping );
if ( 2 == count( $pieces ) )
{
$key = urldecode( $pieces[0] );
$val = urldecode( $pieces[1] );
$map[$key] = $val;
}
if ( defined( "VERBOSE" ) && VERBOSE ) error_log( "Mapping( $mapping ) : $key | $val" );
}
}
return $map;
}
Log to Pub Sub
use Google\Cloud\PubSub\PubSubClient;
function LogToPubSub( $environment, $db_info, $db_version, $sp_call, $response, $delta )
{
if ( "TRUE" == getenv( "USE_GOOGLE_PUB_SUB" ) )
{
$topic = GetPubSubTopic();
$procedure = $sp_call->proc_name . "('" . join( "','", $sp_call->arguments ) . "')";
if ( !$topic )
{
NeoLog( "Not logged to Google PubSub" );
}
else
{
$now = date( "c", time() );
$api_hostname = $environment['server_name' ];
$endpoint = $environment['redirect_url'];
$request = $environment['request' ];
$db = $db_info->dbname . $db_version;
$db_hostname = DB_HOSTNAME;
$sql64 = base64_encode( $procedure );
$ip_address = $_SERVER['SERVER_ADDR'];
$requesting_url = array_get( $request, "wab_requesting_url" );
$cgi_request_method = array_get( $_SERVER , "REQUEST_METHOD" );
$cgi_request_time = array_get( $_SERVER , "REQUEST_TIME" );
$cgi_http_accept = array_get( $_SERVER , "HTTP_ACCEPT" );
$cgi_http_accept_charset = array_get( $_SERVER , "HTTP_ACCEPT_CHARSET" );
$cgi_http_accept_encoding = array_get( $_SERVER , "HTTP_ACCEPT_ENCODING" );
$cgi_http_accept_language = array_get( $_SERVER , "HTTP_ACCEPT_LANGUAGE" );
$cgi_http_referer = array_get( $_SERVER , "HTTP_REFERER" );
$cgi_http_user_agent = array_get( $_SERVER , "HTTP_USER_AGENT" );
$cgi_remote_addr = array_get( $_SERVER , "HTTP_REMOTE_ADDR" );
$cgi_remote_port = array_get( $_SERVER , "HTTP_REMOTE_PORT" );
$cgi_remote_user = array_get( $_SERVER , "HTTP_REMOTE_USER" );
$json = "{\n";
$json .= "\"ts\": \"$now\",\n";
$json .= "\"api_hostname\": \"$api_hostname\",\n";
$json .= "\"endpoint\": \"$endpoint\",\n";
$json .= "\"type\": \"r/w\",\n";
$json .= "\"status\": \"$response->status\",\n";
$json .= "\"error\": \"$response->error\",\n";
$json .= "\"sql64\": \"$sql64\",\n";
$json .= "\"response_time_ms\": \"$response->microsecs\",\n";
$json .= "\"db\": \"$db\",\n";
$json .= "\"db_version\": \"$db_version\",\n";
$json .= "\"db_hostname\": \"$db_hostname\",\n";
$json .= "\"ip_address\": \"$ip_address\",\n";
$json .= "\"requesting_url\": \"$requesting_url\",\n";
$json .= "\"cgi_request_method\": \"$cgi_request_method\",\n";
$json .= "\"cgi_request_time\": \"$cgi_request_time\",\n";
$json .= "\"cgi_http_accept\": \"$cgi_http_accept\",\n";
$json .= "\"cgi_http_accept_charset\": \"$cgi_http_accept_charset\",\n";
$json .= "\"cgi_http_accept_encoding\": \"$cgi_http_accept_encoding\",\n";
$json .= "\"cgi_http_accept_language\": \"$cgi_http_accept_language\",\n";
$json .= "\"cgi_http_referer\": \"$cgi_http_referer\",\n";
$json .= "\"cgi_http_user_agent\": \"$cgi_http_user_agent\",\n";
$json .= "\"cgi_remote_addr\": \"$cgi_remote_addr\",\n";
$json .= "\"cgi_remote_port\": \"$cgi_remote_port\",\n";
$json .= "\"cgi_remote_user\": \"$cgi_remote_user\"\n";
$json .= "}\n";
try
{
$topic->publish( ['data' => $json] );
NeoLog( "Logged to Google PubSub" );
}
catch ( Exception $ex )
{
NeoLog( $ex );
}
}
}
}
function GetPubSubTopic()
{
$topic = null;
NeoLogIn();
if ( !getenv( "GOOGLE_APPLICATION_CREDENTIALS" ) )
{
NeoLog( "!!! WARNING !!! GOOGLE_APPLICATION_CREDENTIALS not defined in Apache configuration, but USE_GOOGLE_PUB_SUB is!" );
}
else
if ( !getenv( "GOOGLE_PUB_SUB_PROJECT" ) )
{
NeoLog( "!!! WARNING !!! GOOGLE_PUB_SUB_PROJECT not defined in Apache configuration, but USE_GOOGLE_PUB_SUB is!" );
}
else
if ( !getenv( "GOOGLE_PUB_SUB_TOPIC" ) )
{
NeoLog( "!!! WARNING !!! GOOGLE_PUB_SUB_TOPIC not defined in Apache configuration, but USE_GOOGLE_PUB_SUB is!" );
}
else
{
$pubsub_project = getenv( "GOOGLE_PUB_SUB_PROJECT" );
$pubsub_topic = getenv( "GOOGLE_PUB_SUB_TOPIC" );
try
{
$pubsub = new PubSubClient( ['projectId' => $pubsub_project] );
$topic = $pubsub->topic( $pubsub_topic );
}
catch( Exception $e )
{
NeoLog( "!!! WARNING !!! Could not use GOOGLE credentials: " . getEnv( "GOOGLE_APPLICATION_CREDENTIALS" ) );
}
}
NeoLogOut();
return $topic;
}
A -- Strings
<?php
function array_get( $array, $key )
{
return array_key_exists( $key, $array ) ? $array[$key] : "";
}
function string_contains( $haystack, $needle )
{
return (0 == strlen($needle)) || (false !== strpos( $haystack, $needle ));
}
function string_has_prefix( $haystack, $needle )
{
return (0 == strlen($needle)) || (0 === strpos( $haystack, $needle ));
}
function string_has_suffix( $haystack, $needle )
{
$expected = strlen( $haystack ) - strlen( $needle );
return (0 == strlen($needle)) || ($expected === strrpos( $haystack, $needle ));
}
function test()
{
if ( string_contains( "www.example.com", "" ) ) echo "Passed #1" . "\n";
if ( string_contains( "www.example.com", "www." ) ) echo "Passed #2" . "\n";
if ( string_contains( "www.example.com", ".example." ) ) echo "Passed #3" . "\n";
if ( string_contains( "www.example.com", ".com" ) ) echo "Passed #4" . "\n";
if ( !string_contains( "www.example.com", "wwl" ) ) echo "Passed #5" . "\n";
if ( !string_contains( "www.example.com", ".exanple" ) ) echo "Passed #6" . "\n";
if ( !string_contains( "www.example.com", ".con" ) ) echo "Passed #7" . "\n";
if ( string_has_prefix( "www.example.com", "" ) ) echo "Passed #8" . "\n";
if ( string_has_prefix( "www.example.com", "www." ) ) echo "Passed #9" . "\n";
if ( string_has_prefix( "www.example.com", "www.example." ) ) echo "Passed #A" . "\n";
if ( string_has_prefix( "www.example.com", "www.example.com" ) ) echo "Passed #B" . "\n";
if ( !string_has_prefix( "www.example.com", "wwl" ) ) echo "Passed #C" . "\n";
if ( !string_has_prefix( "www.example.com", "www.exanple" ) ) echo "Passed #D" . "\n";
if ( !string_has_prefix( "www.example.com", "www.example.con" ) ) echo "Passed #E" . "\n";
if ( string_has_suffix( "www.example.com", "" ) ) echo "Passed #F" . "\n";
if ( string_has_suffix( "www.example.com", ".com" ) ) echo "Passed #G" . "\n";
if ( string_has_suffix( "www.example.com", ".example.com" ) ) echo "Passed #H" . "\n";
if ( string_has_suffix( "www.example.com", "www.example.com" ) ) echo "Passed #I" . "\n";
if ( !string_has_suffix( "www.example.com", ".con" ) ) echo "Passed #J" . "\n";
if ( !string_has_suffix( "www.example.com", ".example.con" ) ) echo "Passed #K" . "\n";
if ( !string_has_suffix( "www.example.com", "www.example.con" ) ) echo "Passed #L" . "\n";
}
<?php
class CSV
{
static $newline = "\r\n";
static function PlainText()
{
header( "Content-Type: text/plain" );
}
static function InitiateDownload( $filename="download.csv" )
{
header( "Content-Type: application/octet-stream" );
header( "Content-Disposition: attachment; filename=\"$filename\"" );
header("Pragma: public");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
}
static function encode( $results, $numbered = true, $double_spaced = false )
{
$ret = "";
if ( is_array( $results ) && (0 < count( $results ) ) )
{
$ret .= CSV::Heading( $results, $numbered );
$ret .= CSV::Rows( $results, $numbered, $double_spaced );
}
return $ret;
}
static function Echo( $results, $numbered = true, $double_spaced = false )
{
if ( is_a( $results, 'Results' ) )
{
$it = $results->iterator();
if ( $it->hasNext() )
{
NeoLog( "CSV::Echo" );
CSV::EchoHeading( $it, $numbered );
CSV::EchoRows ( $it, $numbered, $double_spaced );
}
}
}
static function EchoHeading( $it, $numbered )
{
$heading = "";
if ( $it->hasFirst() )
{
$tuple = $it->getFirst();
$sep = "";
foreach ( $tuple as $key => $value )
{
$heading .= $sep;
$heading .= $key;
$sep = ",";
}
unset( $tuple ); $tuple = null;
}
if ( $numbered ) $heading .= ",#";
echo $heading .= self::$newline;
}
static function Heading( $it, $numbered )
{
$heading = "";
foreach ( $results as $tuple )
{
$sep = "";
foreach ( $tuple as $key => $value )
{
$heading .= $sep;
$heading .= $key;
$sep = ",";
}
break;
}
if ( $numbered ) $heading .= ",#";
return $heading .= self::$newline;
}
static function EchoRows( $it, $numbered, $double_spaced = false )
{
$nr = 0;
while ( $it->hasNext() )
{
$tuple = $it->next();
$row = "";
$sep = "";
$nr++;
if ( $tuple )
{
foreach ( $tuple as $key => $value )
{
$row .= $sep;
$value = html_entity_decode( $value, ENT_QUOTES );
if ( string_contains( $value, "," ) )
{
$row .= "\"$value\"";
}
else
{
$row .= "$value";
}
$sep = ",";
}
if ( $numbered ) $row .= ",$nr";
if ( $double_spaced ) $row .= self::$newline;
}
$row .= self::$newline;
echo $row;
unset( $tuple ); $tuple = null;
flush();
}
}
static function Rows( $results, $numbered, $double_spaced = false )
{
$nr = 0;
$rows = "";
foreach ( $results as $tuple )
{
$nr++;
$sep = "";
foreach ( $tuple as $key => $value )
{
$rows .= $sep;
$value = html_entity_decode( $value, ENT_QUOTES );
if ( string_contains( $value, "," ) )
{
$rows .= "\"$value\"";
}
else
{
$rows .= "$value";
}
$sep = ",";
}
if ( $numbered ) $rows .= ",$nr";
$rows .= self::$newline;
if ( $double_spaced ) $rows .= self::$newline;
}
return $rows;
}
static function MapValues( $map, $tuples )
{
$mapped_tuples = array();
foreach ( $tuples as $obj )
{
$tuple = (array) $obj;
$mapped = array();
foreach ( $map as $label => $key )
{
if ( array_key_exists( $key, $tuple ) )
{
$mapped[$label] = $tuple[$key];
}
else
{
error_log( "Could not find $key in tuple." );
}
}
$mapped_tuples[] = $mapped;
}
return $mapped_tuples;
}
}
Experimental
The experimental function is used to determine if experimental code paths should be followed for the current domain.
<?php
// Copyright (c) 2009, 2020 Daniel Robert Bradley. All rights reserved.
// This software is distributed under the terms of the GNU Lesser General Public License version 2.1
?>
<?php
function Experimental()
{
return
("TRUE" == getenv( "USE_EXPERIMENTAL"))
||
(defined( "EXPERIMENTAL" ) && array_key_exists( $_SERVER["SERVER_NAME"], EXPERIMENTAL ) && EXPERIMENTAL[$_SERVER["SERVER_NAME"]]);
}
Input
<?php
// Copyright (c) 2009, 2010 Daniel Robert Bradley. All rights reserved.
// This software is distributed under the terms of the GNU Lesser General Public License version 2.1
?>
<?php
class Output
{
function println()
{}
function indent()
{}
function outdent()
{}
}
function DBi_escape( $string )
{
return $string;
$db = DBi_anon();
if ( $db->connect( new NullPrinter() ) )
{
return $db->escape( $string );
}
}
class Input
{
static function FilterInput( $request, $debug )
{
$debug = new Output();
$filtered = array();
$debug->println( "<!-- FilterInput() start -->" );
$debug->indent();
{
$debug->println( "<!-- REQUEST -->" );
$debug->indent();
{
foreach ( $request as $key => $val )
{
$filtered_key = Input::Filter( $key );
$filtered_val = Input::Filter( $val );
$filtered[$filtered_key] = $filtered_val;
if ( is_array( $filtered_val ) )
{
$debug->println( "<!-- \"$filtered_key\" | Array -->" );
}
else
{
$debug->println( "<!-- \"$filtered_key\" | \"$filtered_val\" -->" );
}
}
}
$debug->outdent();
$debug->println( "<!-- COOKIE -->" );
$debug->indent();
{
foreach ( $_COOKIE as $key => $val )
{
if ( ! array_key_exists( $key, $filtered ) )
{
$filtered_key = Input::Filter( $key );
$filtered_val = Input::Filter( $val );
$filtered[$filtered_key] = $filtered_val;
$debug->println( "<!-- \"$filtered_key\" | \"$filtered_val\" -->" );
}
}
}
$debug->outdent();
}
$debug->outdent();
$debug->println( "<!-- FilterInput() end -->" );
return $filtered;
}
static function Filter( $value )
{
if ( is_array( $value ) )
{
$ret = array();
foreach ( $value as $key => $val )
{
$filtered_key = Input::Filter( $key );
$filtered_val = Input::Filter( $val );
$ret[$filtered_key] = $filtered_val;
}
return $ret;
}
else
if ( is_string( $value ) )
{
$value = Input::unidecode( $value );
$value = htmlspecialchars( $value, ENT_QUOTES, 'UTF-8', false );
$value = addslashes( $value );
$value = DBi_escape( $value );
$value = str_replace( "\n\r", "<br>", $value );
$value = str_replace( "\n", "<br>", $value );
$value = str_replace( "\\\\", "\", $value );
$value = str_replace( "\x09", " ", $value );
return $value;
}
else
if ( is_null( $value ) )
{
return "";
}
else
{
error_log( "Input::Filter( $value ): unexpected value!" );
}
}
static function unidecode( $value )
{
$str = "";
$n = strlen( $value );
$i = 0;
while ( $i < $n )
{
$ch = substr( $value, $i, 1 );
$val = ord( $ch );
if ( ($val == (0xFC | $val)) && ($i+5 < $n) ) // 6 byte unicode
{
$str .= Input::utf2html( substr( $value, $i, 6 ) );
$i += 6;
}
else
if ( ($val == (0xF8 | $val)) && ($i+4 < $n) ) // 5 byte unicode
{
$str .= Input::utf2html( substr( $value, $i, 5 ) );
$i += 5;
}
else
if ( ($val == (0xF0 | $val)) && ($i+3 < $n) ) // 4 byte unicode
{
$str .= Input::utf2html( substr( $value, $i, 4 ) );
$i += 4;
}
else
if ( ($val == (0xE0 | $val)) && ($i+2 < $n) ) // 3 byte unicode
{
$str .= Input::utf2html( substr( $value, $i, 3 ) );
$i += 3;
}
else
if ( ($val == (0xC0 | $val)) && ($i+1 < $n) ) // 2 byte unicode
{
$str .= Input::utf2html( substr( $value, $i, 2 ) );
$i += 2;
}
else
if ( $val == (0x80 | $val) ) // extra byte
{
error_log( "Warning detected invalid unicode" );
$str .= '?';
$i++;
}
else // ascii character
{
$str .= $ch;
$i++;
}
}
return $str;
}
static function utf2html( $string )
{
$array = Input::utf8_to_unicode( $string );
$string = Input::unicode_to_entities( $array );
return $string;
}
static function utf8_to_unicode( $str )
{
$unicode = array();
$values = array();
$lookingFor = 1;
for ($i = 0; $i < strlen( $str ); $i++ ) {
$thisValue = ord( $str[ $i ] );
if ( $thisValue < 128 ) $unicode[] = $thisValue;
else {
if ( count( $values ) == 0 ) $lookingFor = ( $thisValue < 224 ) ? 2 : 3;
$values[] = $thisValue;
if ( count( $values ) == $lookingFor ) {
$number = ( $lookingFor == 3 ) ?
( ( $values[0] % 16 ) * 4096 ) + ( ( $values[1] % 64 ) * 64 ) + ( $values[2] % 64 ):
( ( $values[0] % 32 ) * 64 ) + ( $values[1] % 64 );
$unicode[] = $number;
$values = array();
$lookingFor = 1;
} // if
} // if
} // for
return $unicode;
}
static function unicode_to_entities( $unicode )
{
$entities = '';
foreach( $unicode as $value ) $entities .= '&#' . $value . ';';
return $entities;
}
}
<?php
// Copyright (c) 2014 Daniel Robert Bradley. All rights reserved.
// This software is distributed under the terms of the GNU Lesser General Public License version 2.1
?>
<?php
class JSON4
{
static function EncodeResults( $indent, $results )
{
$json = "";
$json .= "{";
if ( is_array( $results ) && (0 < count( $results ) ) )
{
$json .= '"results":';
$json .= "[";
$sep = "";
foreach ( $results as $tuple )
{
$json .= $sep;
$json .= self::EncodeTuple( $indent + 1, $tuple );
$sep = ",";
}
$json .= "]";
}
$json .= "}";
return $json;
}
static function EncodeTuple( $indent, $tuple )
{
$json = "\n" . str_repeat( " ", $indent * 4 );
$json .= "{";
$sep = "";
foreach ( $tuple as $key => $value )
{
$json .= $sep;
$json .= self::EncodeStringValue( $indent + 1, $key, $value );
$sep = ",";
}
$json .= "\n" . str_repeat( " ", $indent * 4 ) . "}";
return $json;
}
static function Encode( $indent, $something )
{
$json = "" . str_repeat( " ", $indent * 4 );
if ( is_array( $something ) )
{
$json .= self::EncodeArray( $indent + 1, $something );
}
else
{
$json .= self::EncodeObject( $indent + 1, $something );
}
return $json;
}
static function EncodeArray( $indent, $array )
{
$json = "\n" . str_repeat( " ", $indent * 4 );
$json .= "[";
$sep = "";
if ( self::is_assoc( $array ) )
{
foreach ( $array as $string => $value )
{
$json .= $sep . "{";
$json .= self::EncodeStringValue( $indent + 1, $string, $value );
$json .= "}";
$sep = ",";
}
}
else
{
foreach ( $array as $value )
{
$json .= $sep;
$json .= self::EncodeValue( $indent + 1, $value );
$sep = ",";
}
}
$json .= "\n" . str_repeat( " ", $indent * 4 ) . "]";
return $json;
}
static function EncodeObject( $indent, $object )
{
$json = "\n" . str_repeat( " ", $indent * 4 );
$json .= "{";
$sep = "";
foreach ( $object as $member => $value )
{
$json .= $sep;
$json .= self::EncodeStringValue( $indent + 1, $member, $value );
$sep = ",";
}
$json .= "\n" . str_repeat( " ", $indent * 4 ) . "}";
return $json;
}
static function EncodeStringValue( $indent, $string, $value )
{
$json = "\n" . str_repeat( " ", $indent * 4 );
$json .= self::EncodeString( $indent + 1, $string );
$json .= ":";
$json .= self::EncodeValue( $indent + 1, $value );
return $json;
}
static function EncodeString( $indent, $string )
{
$json = "";
$escaped = str_replace( "\\", "\\\\", $string );
$json .= "\"$escaped\"";
return $json;
}
static function EncodeValue( $indent, $value )
{
$json = "";
if ( is_array( $value ) )
{
$json .= self::EncodeArray( $indent, $value );
}
else
if ( is_string( $value ) )
{
$json .= self::EncodeString( $indent, $value );
}
else
if ( is_numeric( $value ) )
{
$json .= self::EncodeNumber( $indent, $value );
}
else
if ( is_null( $value ) )
{
$json .= "null";
}
else
if ( true === $value )
{
$json .= "true";
}
else
if ( false === $value )
{
$json .= "false";
}
else
{
$json .= self::EncodeObject( $indent, $value );
}
return $json;
}
static function EncodeNumber( $indent, $number )
{
$json = "";
$json .= $number;
return $json;
}
static function is_assoc( $a )
{
$assoc = true;
$keys = array_keys( $a );
foreach ( $keys as $key )
{
if ( is_numeric( $key ) && (0 == $key) ) $assoc = false;
break;
}
return $assoc;
}
static function is_assocx( $a )
{
$b = array_keys($a);
return ($a != array_keys($b));
}
}
<?php
// Copyright (c) 2015 Daniel Robert Bradley. All rights reserved.
// This software is distributed under the terms of the GNU Lesser General Public License version 2.1
?>
<?php
class JStream
{
static function Encode( $indent, $something )
{
if ( is_array( $something ) )
{
self::EncodeArray( $indent, $something );
}
else
{
self::EncodeObject( $indent, $something );
}
}
static function EncodeResults( $indent, $results )
{
echo "{";
if ( is_array( $results ) && (0 < count( $results ) ) )
{
echo '"results":';
echo "[";
$sep = "";
foreach ( $results as $tuple )
{
echo $sep;
self::EncodeTuple( $index + 1, $tuple );
$sep = ",";
}
echo "]";
}
echo "}";
}
static function EncodeTuple( $indent, $tuple )
{
echo "\n" . str_repeat( " ", $indent * 4 ) . "{";
$sep = "";
foreach ( $tuple as $key => $value )
{
echo $sep;
self::EncodeStringValue( $indent + 1, $key, $value );
$sep = ",";
}
echo "\n" . str_repeat( " ", $indent * 4 ) . "}";
}
static function EncodeArray( $indent, $array )
{
echo "\n" . str_repeat( " ", $indent * 4 );
echo "[";
$sep = "";
if ( self::is_assoc( $array ) )
{
foreach ( $array as $string => $value )
{
echo $sep . "{";
self::EncodeStringValue( $indent + 1, $string, $value );
echo "}";
$sep = ",";
}
}
else
{
foreach ( $array as $value )
{
echo $sep;
self::EncodeValue( $indent + 1, $value );
$sep = ",";
}
}
echo "\n" . str_repeat( " ", $indent * 4 );
echo "]";
}
static function EncodeObject( $indent, $object )
{
echo "\n" . str_repeat( " ", $indent * 4 );
echo "{";
$sep = "";
foreach ( $object as $member => $value )
{
echo $sep;
self::EncodeStringValue( $indent + 1, $member, $value );
$sep = ",";
}
if ( property_exists( $object, "results" ) )
{
$results = $object->results;
//
// This must happen _after_ the results have been sent.
//
$count = is_array( $results ) ? count( $results ) : (is_a( $results, 'Results' ) ? $results->numRows() : '?');
echo $sep . "\n";
echo str_repeat( " ", ($indent + 1) * 4 );
echo json_encode( "count" );
echo ":";
echo json_encode( $count );
}
echo "\n" . str_repeat( " ", $indent * 4 );
echo "}";
}
static function EncodeStringValue( $indent, $string, $value )
{
echo "\n" . str_repeat( " ", $indent * 4 );
if ( is_string( $value ) )
{
$headers = apache_request_headers();
if ( array_key_exists( 'Accept', $headers ) )
{
switch( $headers['Accept'] )
{
case "application/json":
$value = html_entity_decode( $value, ENT_QUOTES );
$value = str_replace( "<br>", "\n", $value );
break;
}
}
}
if ( string_has_prefix( $string, "__json_" ) )
{
self::EncodeString( $indent, substr( $string, 7 ) );
echo ":";
echo json_encode( json_decode( $value ) );
}
else
{
self::EncodeString( $indent, $string );
echo ":";
self::EncodeValue( $indent, $value );
}
}
static function EncodeString( $indent, $string )
{
$escaped = json_encode( $string ); // str_replace( "\\", "\\\\", $string );
if ( false === $escaped )
{
error_log( "ERROR1: " . var_export( $string, true ) );
$escaped = json_encode( htmlentities( $string ) );
}
if ( false === $escaped )
{
error_log( "ERROR2: " . var_export( $string, true ) );
$escaped = json_encode( "" );
}
echo $escaped;
}
static function EncodeValue( $indent, $value )
{
if ( true === $value )
{
echo "true";
}
else
if ( false === $value )
{
echo "false";
}
else
if ( is_array( $value ) )
{
self::EncodeArray( $indent, $value );
}
else
if ( is_string( $value ) )
{
self::EncodeString( $indent, $value );
}
else
if ( is_numeric( $value ) )
{
self::EncodeNumber( $indent, $value );
}
else
if ( is_null( $value ) )
{
echo "null";
}
else
if ( is_a( $value, 'Results' ) )
{
if ( $value->isProcessed() )
{
// This should never be executed because the 'Results' object
// is only every "processed" by the TranslateToCSV code path
// which is orthogonal to this one...
// I will remove this when I remove the "map" code path
// from TranslateToCSV.
NeoLog( "Abort!!! Invalid code path." );
//$array = $value->toArray();
//self::EncodeArray( $indent, $array );
}
else
{
self::EncodeResultArray( $indent, $value );
}
}
else
{
self::EncodeObject( $indent, $value );
}
}
static function EncodeResultArray( $indent, $result_array )
{
echo "\n" . str_repeat( " ", $indent * 4 );
echo "[";
$sep = "";
$it = $result_array->iterator();
while ( $it->hasNext() )
{
$value = $it->next();
echo $sep;
\JStream::EncodeValue( $indent + 1, $value );
$sep = ",";
unset( $value ); $value = null;
}
echo "\n" . str_repeat( " ", $indent * 4 );
echo "]";
}
static function EncodeNumber( $indent, $number )
{
echo json_encode( $number );
}
static function is_assoc( $a )
{
$assoc = true;
$keys = array_keys( $a );
foreach ( $keys as $key )
{
if ( is_numeric( $key ) && (0 == $key) ) $assoc = false;
break;
}
return $assoc;
}
static function is_assocx( $a )
{
$b = array_keys($a);
return ($a != array_keys($b));
}
}
Log
<?php
class Neo
{
static $neolog_indent;
static $neolog_connection_id;
static $neolog_log_level;
static function Init()
{
$strong;
self::$neolog_connection_id = bin2hex( openssl_random_pseudo_bytes( 2, $strong ) );
self::$neolog_log_level = intval( getenv( "SHOW_LOG_TO_LEVEL" ) );
if ( ! is_numeric( self::$neolog_log_level ) )
{
self::$neolog_log_level = 100;
}
}
static function In()
{
self::$neolog_indent++;
}
static function Out()
{
if ( 0 < self::$neolog_indent ) self::$neolog_indent--;
}
static function Log( $text, $priority = LOG_INFO )
{
$indent = intval( self::$neolog_indent );
$level = intval( self::$neolog_log_level );
if ( $indent <= $level )
{
self::LogError( $text, $priority );
}
}
static function LogError( $text, $priority = LOG_ERR )
{
$now = isset( $_SERVER ) ? "" : date( DATE_ISO8601 );
$stars = str_repeat( "+", self::$neolog_indent );
error_log( sprintf( "[%s] %s%-8s %s", self::$neolog_connection_id, $now, $stars, $text ) );
syslog( $priority, $stars . " " . $text );
}
}
Neo::Init();
function NeoLog( $text, $priority = LOG_INFO )
{
Neo::Log( $text, $priority );
}
function NeoLogError( $text )
{
Neo::LogError( $text, LOG_ERR );
}
function NeoLogIn()
{
Neo::In();
}
function NeoLogOut()
{
Neo::Out();
}
function NeoLog_DecodeFacility( $facility_name )
{
switch( $facility_name )
{
case "AUTH":
return LOG_AUTH;
case "AUTHPRIV":
return LOG_AUTHPRIV;
case "DAEMON":
return LOG_DAEMON;
case "KERN":
return LOG_KERN;
case "LOCAL0":
return LOG_LOCAL0;
case "LOCAL1":
return LOG_LOCAL1;
case "LOCAL2":
return LOG_LOCAL2;
case "LOCAL3":
return LOG_LOCAL3;
case "LOCAL4":
return LOG_LOCAL4;
case "LOCAL5":
return LOG_LOCAL5;
case "LOCAL6":
return LOG_LOCAL6;
case "LOCAL7":
return LOG_LOCAL7;
case "LPR":
return LOG_LPR;
case "MAIL":
return LOG_MAIL;
case "NEWS":
return LOG_NEWS;
case "SYSLOG":
return LOG_SYSLOG;
case "USER":
return LOG_USER;
case "UUCP":
return LOG_UUCP;
default:
return LOG_USER;
}
}
MySQL
<?php
function MySQLCreate()
{
//NeoLog( 3, "DB_HOSTNAME: " . DB_HOSTNAME );
//NeoLog( 3, "DB_USERNAME: " . DB_USERNAME );
//NeoLog( 3, "DB_PASSWORD: " . DB_PASSWORD );
//NeoLog( 3, "SSL_BASE: " . SSL_BASE );
$m = mysqli_init();
if ( !$m )
{
NeoLogError( "Could not initialise 'mysqli'" );
exit;
}
if ( MYSQLI_OPT_READ_TIMEOUT )
{
if ( defined( "USE_MYSQLI_OPT_READ_TIMEOUT" ) )
{
mysqli_options( $m, MYSQLI_OPT_READ_TIMEOUT, USE_MYSQLI_OPT_READ_TIMEOUT );
}
}
else
{
NeoLog( "Read timeout not supported" );
}
ConfigureSSL( $m, SSL_BASE, DB_HOSTNAME );
$connection = @$m->real_connect( DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, null, null, null, MYSQLI_CLIENT_SSL_DONT_VERIFY_SERVER_CERT );
if ( !$connection )
{
if ( "No such file or directory" == $m->error )
{
NeoLogError( "Aborting: could not connect to " . DB_HOSTNAME . " using " . DB_USERNAME . ":" . DB_PASSWORD );
exit;
}
else
if ( "php_network_getaddresses: getaddrinfo failed: Name or service not known" == $m->error )
{
NeoLogError( "Aborting: could not resolve: " . DB_HOSTNAME );
exit;
}
else
{
NeoLogError( "Aborting: unexpected error: " . $m->error );
exit;
}
}
return $m; // Same as $m.
}
function MySQLClose( $m )
{
mysqli_close( $m );
}
function MySQLInfo( $mysql, $sql_query )
{
$tuples = array();
$resource = mysqli_query( $mysql, $sql_query );
if ( !$resource )
{
NeoLog( $mysql->error );
exit;
}
else
{
while ( $row = mysqli_fetch_array( $resource, MYSQLI_ASSOC ) )
{
$tuples[] = $row;
}
}
return $tuples;
}
function MySQLFunction( $db, $function )
{
$ret = null;
$sql = "SELECT $function";
$m = MySQLCreate();
if ( ! mysqli_select_db( $m, $db ) )
{
NeoLog( "Aborting, could not select database! " . $db );
exit;
}
else
{
$resource = mysqli_query( $m, $sql );
if ( True === $resource )
{
$ret = True;
}
else
if( False === $resource )
{
$ret = False;
}
else
if ( $resource )
{
$row = mysqli_fetch_array( $resource, MYSQLI_NUM );
$ret = array_key_exists( 0, $row ) ? $row[0] : NULL;
mysqli_free_result( $resource );
}
else
{
NeoLog( "Aborting, " . $m->error );
exit;
}
mysqli_next_result( $m );
}
MySQLClose( $m );
return $ret;
}
function MySQLProcedure( $db, $sp_call, &$procedure = "" )
{
$ret = False;
if ( !is_object( $sp_call ) )
{
debug_backtrace();
}
$proc_name = $sp_call->proc_name;
$arguments = $sp_call->arguments;
$m = MySQLCreate();
if ( ! mysqli_select_db( $m, $db ) )
{
NeoLog( "Aborting, could not select database! " . $db );
MySQLClose( $m );
exit;
}
else
{
$count = 0;
$filtered = array();
foreach( $arguments as $argument )
{
if ( is_null( $argument ) )
{
$filtered[] = "NULL";
}
else
{
$filtered[] = "'" . mysqli_real_escape_string( $m, $argument ) . "'";
}
}
$procedure = $proc_name . "(" . join( ",", $filtered ) . ")";
$sql_query = "CALL " . $procedure;
if ( "wab_test" == $proc_name )
{
NeoLog( "Test: " . $sql_query );
}
try
{
$resource = mysqli_query( $m, $sql_query );
}
catch ( Exception $e )
{
// MySQL thrown errors seem to be now throwing exceptions...
// Is this a MariaDB thing; or updated phpmysqli...
//
$resource = False;
$ret = $e->getMessage();
}
if ( True === $resource )
{
$ret = array();
}
else
if( False === $resource )
{
$ret = $m->error;
}
else
if ( $resource )
{
$ret = array();
while ( $row = mysqli_fetch_array( $resource, MYSQLI_ASSOC ) )
{
$ret[] = (object) $row;
$count++;
}
mysqli_free_result( $resource );
}
else
{
NeoLogError( "Called: " . $sql_query . ": threw ERROR: " . $m->error );
MySQLClose( $m );
exit;
}
mysqli_next_result( $m );
if ( "TRUE" == getenv( "LOG_STORED_PROCEDURE_CALL" ) )
{
NeoLogError( "Called: " . $sql_query . ": returned " . $count . " rows" );
}
}
MySQLClose( $m );
return $ret;
}
class ResultIterator
{
protected $m;
protected $result;
protected $row;
protected $count = 0;
function __construct( $anM, $aResult )
{
$this->m = $anM;
$this->result = $aResult;
$this->row = mysqli_fetch_array( $this->result, MYSQLI_ASSOC );
$this->first = $this->row;
}
function __destruct()
{
mysqli_free_result( $this->result );
mysqli_next_result( $this->m );
MySQLClose ( $this->m );
}
function hasFirst()
{
return $this->first ? true : false;
}
function getFirst()
{
return (object) $this->first;
}
function hasNext()
{
return $this->row ? true : false;
}
function next()
{
$ret = $this->row;
$this->row = null;
$this->row = mysqli_fetch_array( $this->result, MYSQLI_ASSOC );
$this->count++;
return (object) $ret;
}
function getCount()
{
return $this->count;
}
}
class Results
{
public $sqlQuery;
protected $it;
protected $array;
protected $result;
function __construct( $sql_query, $anM, $aResult )
{
$this->sqlQuery = $sql_query;
$this->it = new ResultIterator( $anM, $aResult );
$this->array = null;
$this->result = $aResult;
}
function hasFirst()
{
return $this->it->hasFirst();
}
function getFirst()
{
return $this->it->getFirst();
}
// Called by OutputResponse.TranslateToCSV
function toArray()
{
if ( is_null( $this->array ) )
{
$this->array = array();
while ( $this->it->hasNext() )
{
$this->array[] = (object) $this->it->next();
}
}
return $this->array;
}
function toStringX()
{
return var_export( $this->array, true );
}
function isProcessed()
{
return !is_null( $this->array );
}
function encodeArrayX( $indent )
{
echo "\n" . str_repeat( " ", $indent * 4 );
echo "[";
$sep = "";
while ( $this->it->hasNext() )
{
$value = (object) $this->it->next();
echo $sep;
\JStream::EncodeValue( $indent + 1, $value );
$sep = ",";
}
echo "\n" . str_repeat( " ", $indent * 4 );
echo "]";
}
function iterator()
{
return $this->it;
}
function numRows()
{
return $this->it->getCount();
//return mysqli_num_rows( $this->result );
}
}
class EmptyResults
{
public $sqlQuery;
protected $array;
function __construct( $sql_query )
{
$this->sqlQuery = $sql_query;
$this->array = array();
}
function numRows()
{
return 0;
}
}
class ErrorResults
{
public $sqlQuery;
public $errorMessage;
function __construct( $sql_query, $error_message )
{
$this->sqlQuery = $sql_query;
$this->errorMessage = $error_message;
}
function numRows()
{
return 'X';
}
}
function MySQLProcedureResults( $db, $sp_call, &$procedure = "" )
{
$ret = False;
if ( !is_object( $sp_call ) )
{
debug_backtrace();
}
$proc_name = $sp_call->proc_name;
$arguments = $sp_call->arguments;
$m = MySQLCreate();
if ( ! mysqli_select_db( $m, $db ) )
{
NeoLog( "Aborting, could not select database! " . $db );
MySQLClose( $m );
exit;
}
else
{
$filtered = array();
foreach( $arguments as $argument )
{
if ( is_null( $argument ) )
{
$filtered[] = "NULL";
}
else
{
$filtered[] = "'" . mysqli_real_escape_string( $m, $argument ) . "'";
}
}
$procedure = $proc_name . "(" . join( ",", $filtered ) . ")";
$sql_query = "CALL " . $procedure;
if ( "wab_test" == $proc_name )
{
NeoLog( "Test: " . $sql_query );
}
try
{
$resource = mysqli_query( $m, $sql_query, MYSQLI_USE_RESULT );
}
catch ( Exception $e )
{
// MySQL thrown errors seem to be now throwing exceptions...
// Is this a MariaDB thing; or updated phpmysqli...
//
$resource = False;
$error_message = $e->getMessage();
$ret = new ErrorResults( $sql_query, $error_message );
}
if ( "auth_login" == strtolower( $proc_name ) )
{
$sql_query = "CALL auth_login( [hidden] )";
}
if ( True === $resource )
{
$ret = new EmptyResults( $sql_query );
}
else
if( False === $resource )
{
$ret = new ErrorResults( $sql_query, $m->error );
}
else
if ( $resource )
{
$ret = new Results( $sql_query, $m, $resource );
}
else
{
NeoLog( "Aborting, " . $m->error );
MySQLClose( $m );
exit;
}
}
if ( !is_a( $ret, 'Results' ) )
{
mysqli_next_result( $m );
MySQLClose( $m );
}
return $ret;
}
SSL
<?php
function ConfigureSSL( $mysqli, $ssl_base, $host )
{
if ( file_exists( "$ssl_base/$host/client-key.pem" ) )
{
mysqli_ssl_set
(
$mysqli,
"$ssl_base/$host/client-key.pem",
"$ssl_base/$host/client-cert.pem",
"$ssl_base/$host/ca-cert.pem",
NULL,
NULL
);
}
else
if ( file_exists( "$ssl_base/$host/server-ca.pem" ) )
{
NeoLog( "Warning: not using x509 authenication as cannot find client key/cert." );
mysqli_ssl_set
(
$mysqli,
NULL,
NULL,
"$ssl_base/$host/server-ca.pem",
NULL,
NULL
);
}
else
if( "localhost" != $host )
{
NeoLog( "Could not locate SSL files: " . "$ssl_base/$host/client-key.pem" );
}
}
Test
DROP PROCEDURE Wab_Test;
DELIMITER //
CREATE PROCEDURE Wab_Test
(
$Sid TEXT,
$apikey CHAR(64),
$date DATE,
$text TEXT,
$number INT
)
SQL SECURITY DEFINER
COMMENT 'EXPORT'
BEGIN
SELECT
$Sid AS Sid,
$apikey AS apikey,
$date AS date,
$text AS text,
$number AS number;
END
//
DELIMITER ;
DROP PROCEDURE Raw_Wab_Test;
DELIMITER //
CREATE PROCEDURE Raw_Wab_Test
(
$Sid TEXT,
$apikey CHAR(64),
$date DATE,
$text TEXT,
$number INT
)
SQL SECURITY DEFINER
COMMENT 'EXPORT'
BEGIN
SELECT
$Sid AS Sid,
$apikey AS apikey,
$date AS date,
$text AS text,
$number AS number;
END
//
DELIMITER ;