Importing data using TT Data File Upload
Overview
TT Trade Surveillance offers the ability to upload data via a .csv file. This allows users to easily add and process data into
TT Trade Surveillance. The TT Trade Surveillance File upload uses the same file upload method that TT uses for uploading credit, start of
day, and product margin files.
Prerequisites
Prior to uploading files:
- You must configure connection access for file uploads. Refer to the Configuring
Access for File Uploads section in the Setup help for more details. - Each company must have at least 1 account with Type set to Audit Trail Only. Refer to Creating a New Account in the Setup help for more details.
Supported Formats
To support this functionality, you must ensure the columns and rows in your .csv file matches one of the
following formats:
-
Generic Audit Trail File: TT’s custom format. When importing data from any external source, you
must ensure the file contents match the correctly mapped Generic Audit Trail File. For your convenience,
you can download a sample mapping file here. -
Trayport Format: TT Trade Surveillance supports .csv files exported directly from the Trayport application. For more information on Trayport, refer to the Trayport’s Daily Market Activity File
Guide available on their website.Note If all of the trader fields (Trader, InitiatorTraderName, AggressorTraderName) are set to empty, those records will be treated as non-customer data and filtered out of Trade Surveillance processing. Also, if one of the trader fields is set to System Anonymous Trader, it will be filtered out of Trade Surveillance processing.
Uploading a File
TT offers two methods to upload data files into the platform: Data File Upload and
Auto Upload. Both methods are available in the Company Settings section of the Setup application.
-
Data File Upload: This method allows direct upload of a single data file.
Note The Data File Upload method has a 10mb file size limit. If
uploading larger amounts of data, TT recommends using the Auto Upload feature. -
Auto Upload: This method allows scheduled, automatic daily file uploads and supports files larger
than 10mb.
Directly uploading a single data file
To upload a direct data file:
-
Ensure that you are logged into the TT platform with an administrator account and open the Setup
application. -
In the Setup application, select Company Settings | Data Uploads in the left navigation
pane.
-
Press the New Upload button. The Upload File window displays.
-
In the Select File Type dropdown, select either Trayport Market Activity File or Generic
Audit Trail File depending on the type of file you wish to upload.
-
For Generic Audit Trail File, select CSV in the Select File Format dropdown menu.
For Trayport Market Activity File, you can select either CSV or, if uploading a compressed
csv file, select ZIP. -
Press the Choose File… button and navigate to the location of your file.
- Optionally, check Continue on Error.
When this option is checked (enabled), Setup continues uploading data regardless of how many upload
errors may be logged and shown in the Upload Errors section. By default, this option is unchecked
(disabled) and the upload stops when the first error is detected. - Press the Upload button to begin the file upload.
A record of the uploaded file and its status appears in the grid. Click the checkbox next to the record to
review the Output section, which displays how many rows of the file are processed. Any errors that occur
during the upload will appear in an Error section on this page.
Uploading via the Auto Upload
As a company administrator, you can use the Auto Uploads tab to schedule recurring, automated data file uploads.
The uploads can be scheduled to occur at the days and times that best suit your company’s business needs.
When creating a schedule, you can set the time (UTC) and days of the week that the upload will run, as well as:
- Select a file format
- Schedule retry attempts
- Select a secured FTP site as the source of the data upload
- Modify existing upload schedules
- Schedule multiple uploads
- View results of the uploads
Setting the File Name for Automatic Uploads
When setting the File Name field, you should set this value equal to
Companyname-%y%m%d(-N).csv.zip where:
- Companyname equals the name of your firm.
- -%y%m%d configures Setup to look for a file with the year, month, and the current date.
- -N represents an optional flag to upload a file dated N days prior.
Setting File Name to Trading_Firm-%y%m%d.csv.zip causes the server to always upload a file with the current date.
While setting File Name to Trading_Firm-%y%m%d(-1).csv.zip causes the server to always upload the previous day’s file.
To schedule automatic uploads:
-
Ensure that you are logged into the TT platform with an administrator account and open the Setup
application. - Click Company Settings | Auto Uploads in the left navigation pane.
- Click + New Auto Upload.
-
In the Auto Upload Settings screen, configure the following (all settings are required unless
noted):-
Name — Enter a name for the scheduled upload. When adding multiple schedules, each
name must be unique. -
Protocol — Select a protocol to use for uploading the data (e.g., SFTP).
-
Server — Enter the server hostname that is the source of the data
upload. The SFTP port is automatically set to “22.” -
Path — Enter the path to the file being uploaded from the server. This field is
optional. -
Username — Sets the username for logging in to the server.
-
Password — Sets the password for logging in to the server.
-
File Name — Enter the name of the file being uploaded.
Set the file name equal to the Companyname-%y%m%d(-N).csv.zip where
Companyname equals the name of your firm and the optional -N flag equals the number of days prior to the
current day. For guidance on setting this value, refer to the Setting the
File Name for Automatic Uploads section above. -
Start Time (UTC): — Sets time at which to upload the file to TT. Enter a time in
UTC in the hh:mm format (e.g., 23:59). -
Retry For (Minutes) — Sets how long to wait before retrying the upload. Enter a
time in minutes. -
Days to Run — Select which days to run the automated upload.
-
File Type — Select the file type (i.e., Trayport Market Activity File or
Generic Audit Trail File). -
File Format — Select the csv format for the selected
file type. -
Continue on error — When this option is checked (enabled), Setup stops logging
after 20 errors, but continues to upload the file. When it is unchecked, Setup stops uploading
data once it logs 20 errors. This option is unchecked (disabled) by
default.
-
-
Click Create.
The schedule appears in the Auto Uploads grid at the top of the screen.
Results of the upload are stored in Data Uploads.
Refer to the Scheduling
Automated Data Uploads section in the Setup help for more information on creating, modifying, and
viewing the results for your automated uploads.
Viewing Automated Upload Results
After the scheduled upload runs, the Setup application displays the results and stores a record of the file
upload on the Data Uploads page. Click Data Uploads in the left-navigation pane to view a log of
the results.
Additional Considerations
Trade Surveillance processing time depends on the size of the data.
At this time, TT does not validate the data. If the data does not appear correctly in TT Trade Surveillance, you should
double-check that
the file’s data correctly matches the Generic Audit Trail File.