Guidance - Seabird colony monitoring data for contractors
Instructions for contractors with specification to manage NatureScot seabird breeding colony monitoring data and the Seabird Monitoring Programme (SMP)
Updated September 2025
Contents
- Introduction
- Alignment with Seabird Monitoring Programme (SMP) site codes
- Count data format
- Metadata for images and videos
- Linking data between counts and media files
- Data Quality Assurance
- Data submission to the SMP online database
Introduction
The purpose of this guidance is to outline the data and rich media requirements to be followed by
NatureScot contractors when undertaking seabird colony or productivity survey activities.
Data collected for each species should follow field data collection methods outlined in the Seabird
monitoring handbook for Britain and Ireland. A summary of the recommended field guide is available on the SMP, which includes recommended field survey methods, count, units and times of year and day that should be followed for each species.
Colony count and productivity data collected under contract from NatureScot should be submitted to the Seabird Monitoring Programme (SMP) in it's final form.
Please note that this guidance is revised occasionally to reflect developments to the data management systems and reflect relevant NatureScot workflows.
Alignment with Seabird Monitoring Programme (SMP) site codes
Sites should be aligned with SMP site codes whenever possible to ensure data is being collected at locations which align with monitoring sites established within the SMP. This will ensure that the data is compatible with the SMP, comparable with previous counts and the data collected can contribute to national monitoring. Therefore, the monitoring team(s) should check the SMP online portal mapping tool to identify the monitoring site within the area(s) of interest and identify the protected area the SMP monitoring sites are within.
Definitions:
- Protected area: A designated protected area as listed in NatureScot SiteLink.
- PA code: A unique code assigned to each protected area by NatureScot.
- SMP Master site: A site name which is used to group SMP monitoring sites together. The master sites can be found on the SMP portal when signed in as a user
- SMP Site: A defined section or coast or inland area where long-term monitoring data is collected. These sites are defined by the SMP and ensure constancy and comparability across multiple survey years. Each SMP site has a unique numeric code. The sites codes, as well as the start and end points, can be found on the SMP portal when signed in as a user.
- Whole-colony count: Where a successful colony count of the whole site is completed. If only a section of the site was surveyed, then this would be a ‘partial’ count.
- Productivity plot: Plot within a colony which will be revisited for a productivity count.
For each of the priority survey areas i.e. protected area specified in the project specification, the protected area details should be compiled along with the associated SMP monitoring sites within the protected area. The protected area names and codes are available on SiteLink, and the associated SMP site names, master sites and codes can be determined from the SMP portal mapping tool.
The information that should be compiled should include:
- Protected area (PA) official name, NatureScot code and EU code
- SMP Master site name, site name and site code
- Whether the site is already registered by others
PA name | PA NS code | PA EU code | SMP Master site | SMP site name | SMP site code | Already registered? |
|---|---|---|---|---|---|---|
| Hoy SPA | 8513 | UK9002141 | Hoy SPA (Master) | Hoy RSPB Reserve | 98991 | Yes |
| Hoy SPA | 8513 | UK9002141 | Hoy SPA (Master) | Rackwick Beach | 110862 | No |
Note each protected areas are likely to contain several SMP monitoring site which may be species-specific. It important to understand the distinction between species-specific sites, and species group sites; for instance, skua & greater black-backed gull inland sites, and ensure that species count data are assigned to the most appropriate site code within the SMP.
Where necessary, please contact BTO to confirm the appropriate site codes and the site boundaries to be used - [email protected]
The SMP database can only accept a single count per site per survey season. The monitoring team should check whether any of the listed monitoring sites have been registered to be monitored by others. In the unlikely event a monitoring site is already registered by others, the team should undertake the count, submit the data to NatureScot and include in the project report, but NOT submit the data to the SMP portal. Checking whether another surveyor is registered to the monitoring site can be done from the SMP online mapping tool.
Before field monitoring activity is started, the list (and where possible, accompanying maps) of the compiled SMP monitoring sites with the associated protected area name and codes should be provided to the NatureScot project manager with the 2025 field work plan for quality assurance checks, and any requested changes should be made.
Count data format
The data should be collected in a format and using codes and vocabularies compatible with the SMP database. Below are the key data fields that should be collected for each count record.
SMP Recording forms are available to download from the BTO website. However, please note the additional fields highlighted below that need to be included in the final dataset provided to NatureScot.
Colony count
| Field name | Definition | Vocabulary definitions |
|---|---|---|
| SMP_RecordID |
| - |
| Species | Standard species name | Follow standard naming conventions outlined in the Monitoring Handbook. Field data collection codes should be translated to full English species names. (see list below) |
| SMP_Country | This is generated by the SMP database when data is submitted and exported | - |
| SMP_County | This is generated by the SMP database when data is submitted and exported | - |
| SMP_SiteCode |
| BTO - Seabirds |
| SMP_SiteName | As defined on the SMP online portal | BTO - Seabirds |
| SMP_MasterSite | As defined on the SMP online portal | BTO - Seabirds |
| SMP_PlotName | Only needed where the survey is not a Whole Colony Count | A sub-section of a site where a successful count was completed. If the whole site was surveyed, then this would be a ‘whole colony count |
| SMP_Start_Grid |
|
|
| SMP_End_Grid |
|
|
| Protected_Area_Name |
| |
| Protected_Area_NSCode |
| |
| Protected_Area_EUCode |
| |
| StartDate | Date of field survey activity | dd/mm/yyyy |
| EndDate | If appropriate, this is the end date of the survey for multi-day survey activities. | dd/mm/yyyy |
| Time | Start to end times of the field survey activity. | hh:mm – hh:mm |
| Method | The method used for the count, using the codes listed | The full list of methods to be used for each species and the associated codes are outlined in the SMP field guide |
| Count_unit |
|
|
| Count |
| |
| Accuracy |
| |
| EstimateType |
| |
| Visibility |
| |
| Seastate |
| |
| Precipitation | Description of precipitation or rain which would affect visibility of the survey site. |
|
| Wind |
|
|
| Comments |
| |
| MarineOrnith_ProjectID |
|
|
| Recorder | Identify the surveyor(s) who undertook the count | |
| Start_Eastings |
| In BNG or WGS84 |
| Start_Northings |
| In BNG or WGS84 |
| End_Eastings |
| In BNG or WGS84 |
| End_Northings |
| In BNG or WGS84 |
Productivity / Breeding success
| Field name | Definition | Vocabulary definitions |
|---|---|---|
| SMP_RecordID | This is a persistent unique identifier from the SMP database for each record. This ID will also be included in the metadata for imagery collected during the survey. | - |
| Species | Standard species name | Follow standard naming conventions outlined in the Handbook. Field data collection codes should be translated to full English species names. |
| SMP_Country | This is generated by the SMP database when data is submitted and exported | - |
| SMP_County | This is generated by the SMP database when data is submitted and exported | - |
| SMP_SiteCode | As defined on the SMP online portal This is generated by the SMP database when data is submitted and exported | BTO - Seabirds |
| SMP_SiteName | As defined on the SMP online portal | BTO - Seabirds |
| SMP_MasterSite | As defined on the SMP online portal | BTO - Seabirds |
| SMP_PlotName | The section of the site which was re-visited for monitoring of breeding success. | - |
| SMP_Start_Grid |
|
|
| SMP_End_Grid |
|
|
| Protected_Area_Name |
| - |
| Protected_Area_NSCode |
| - |
| Protected_Area_EUCode |
| - |
| StartDate | Date of field survey activity | dd/mm/yyyy |
| Time | Start to end times of the field survey activity. | hh:mm – hh:mm |
| Method | The method used for the count, using the codes listed | The full list of methods to be used for each species and the associated codes are outlined in the SMP field guide |
| Count_unit |
|
|
| Count |
| - |
| CountAccuracy | - |
|
| FledgedCount |
| - |
| FledgedAccuracy | - |
|
| Comments | Any relevant observations about the site or count that sound be noted. | - |
| MarineOrnith_ProjectID |
|
|
| Recorder | Identity the surveyor(s) who undertook the count | - |
| Start_Eastings |
| In BNG or WGS84 |
| Start_Northings |
| In BNG or WGS84 |
| End_Eastings |
| In BNG or WGS84 |
| End_Northings |
| In BNG or WGS84 |
Metadata for images and videos
NatureScot has an asset management system for rich content storage called the Digital Asset Management Systems (DAMS). Items of rich content associated with surveys may be stored in the DAMS but must have rigorous metadata associated with each file in to order to be properly stored.
The purpose of this section is to assist Marine Ornithology NatureScot contractors in preparing still image and video files for upload and storage in NatureScot’s DAMS.
Semi-/automated methods for processing still image and video files and associated metadata ready for upload into the DAMS have been developed, to make managing the large volumes of rich media collected on marine surveys as efficient as possible and compliant with Marine Environmental Data and Information Network (MEDIN) data standards.
Contractors should only undertake the steps (boxes) involved in preparing media files and metadata for uploading into the DAMS that are highlighted blue in stages 1 and 2 of the summary diagram (Figure 1), the full workflow diagram is included for context only.
The step-by-step guidance for video and still image processing and any requirements involved in executing these two stages (1 and 2) are explained in detail in the following sections:
- Re-naming media files
- Metadata - populating the DAMS spreadsheets
Context for media data requirements
This media workflow aims to help streamline existing NatureScot data management workflows; from the acquisition of video and stills during survey work through to biological analysis, QA and eventual storage in the DAMS with correct attribution of MEDIN compliant metadata to each media file.
The MEDIN data standards are important in helping NatureScot achieve objectives for more integrated data management and sharing, as well as compliance with the INSPIRE Directive to make data re-usable and accessible to the public. Therefore, configuring the DAMS to align with MEDIN data standards is considered an essential part to mobilising NatureScot’s data effectively; ensuring that media are described in a consistent way and that they can be discovered and re-used.
[insert image: 'Summary diagram of the proposed marine image (video and stills) workflow – data acquisition to data storage']
Stage 1: Renaming media files
Some filenames can be non-unique, and the original names will not be compatible with the DAMS image naming convention.
| Examples of old filenames | Examples of new file names |
|---|---|
| F:\NatureScot\98226\Stills\DSC1234.jpg | 98226-2015-06-01_DSC1234.jpg |
| F:\NatureScot\98226\Stills\DSC2345.avi | 98226-2015-06-01_DSC1234.avi |
- Still images require renaming prior to incorporation within DAMS.
- Video files require renaming can use the following renaming process. Is video files are being segmented, they are re-named during the video segmentation process.
There are many ways to rename multiple files including the use of compiled executable programs and the use of scripts within environments such as MS-DOS or Python. Below you will find details outlining the steps involved with different approaches to renaming rich media files.
Using a third party programme
The easiest way of renaming files if the avoidance of scripting is preferred is to use a free program such as Bulk Rename Utility or Flash Renamer. These can both import comma delimited (csv) text files containing two columns, the first being the existing name and the second the new name (see examples in Table 1).
The old names can contain full paths, so that if some of the files have identical names, they can still be renamed as long as they are in different directories. Bulk Rename Utility is perhaps the better of the two programs as you can opt to rename copies of the original files and place them in a new directory, leaving the old files intact. Flash Renamer renames the files in situ.
Using Bulk Rename Utility to rename image files
This program is very good but is not overly intuitive on initial exposure, but these instructions should facilitate its use. Creation of the text file listing the old and new names is in fact the same for Bulk Rename Utility and Flash Re-namer.
The media files are usually spread amongst several directories, so the first job is to produce a list of all the old filenames together with their full paths, for subsequent import into Excel. The easiest way of doing this is to use the command prompt within Windows. This should be run within the parent directory of all the directories containing the files to be renamed. To do this run Windows Explorer and select (left click) in the left-hand pane the directory containing all the image directories. Left click on the address bar to the right of the displayed address text and this will select the highlighted address (Figure 2). Now type CMD and the command prompt window will run within the current address. Type the following (but modify accordingly) into the command window:
Dir /b /s *.JPG > filelist.txt
To get rid of the command window type EXIT.
This places a list of all the JPG image files (with full path names) in all the directories below the current one into the text file “filelist.txt”. This can then be imported into column A of an Excel spreadsheet. Column B of the spreadsheet should contain the new filenames (Table 1).
For DAMS image files this should be MarineOrnithProjectCode_oldfilename; e.g. 98226_2015-06-01_DSC1234.jpg
- Note that underscores separate the elements, but hyphens punctuate the date.
- If the day of the survey is unknown, month and year are adequate.
The new filenames can be created within Excel using concatenation provided the spreadsheet contains columns for the component elements. As a final step the old and new filename lists are exported as two columns in a csv file with no headers (Table 1).
- Run Bulk Rename Utility and import the csv file by selecting the following menu options: Actions > Import Rename-Pairs > Import Rename-Pairs
- Ensure that the directory that includes all the image files or includes all the subdirectories that include the files is selected in the file tree window (top left) (Figure 3).
- All files within the directory will be listed in the file list window (to the right) if folders, files and subfolders have been selected in the Filters(12) pane.
- Select all the files by: Actions > Select All
- Check that in the Filters pane (12) the folders, files and subfolders boxes have been ticked. Also apply a filter to select only those files that you want to rename (e.g. *.JPG or *.CR2).
- The files list window (top right) should now just list the filtered files. Select all of these in the window with the mouse, whereupon the Rename button (bottom right) will become active (ungreyed).
- If you wish to copy renamed versions of the files to a new directory and leave the originals intact, then enter the location of the target directory in pane13 and check the Copy not Move box.
You can now check what the result of renaming will look like by clicking the Preview button. Once happy, click the Rename button.
Using Python to bulk rename files
The below script is designed to bulk rename files. Add the directory, project name, and dst variable to tailor the filename to the appropriate format.
# Python 3 rename multiple files in a directory or folder
# importing modules
import os
from os.path import join
from datetime import datetime
import shutil
from os import walk #Necessary for looking into subfolders
from pathlib import Path
# Function used in file metadata conversion
def dateConvert(atime):
newtime = datetime.fromtimestamp(atime)
return newtime.date()
# Function to rename multiple files
folder = r" ” # add directory here
backup_folder = os.path.split(folder)[0]+"\\Backup" # Go up one level and make directory called \\Backup
proj_no_title = r" " # add project title here
# This section creates a backup subfolder in case name / undo rename is not correct
sour = folder
dest = backup_folder
if os.path.exists(dest):
shutil.rmtree(dest)
shutil.copytree(sour,dest)
# End backup
# Rename function: Walk through subfolders and then enumerate
def rename():
Paths = [join(root, f).replace(os.sep,"/") for root, dirs, files in walk(folder, topdown=True) for f in files]
for count, filename in enumerate(Paths):
# get filename and date modified for each file
stats = os.stat(filename)
date = dateConvert(stats.st_mtime)
file = os.path.basename(filename)
path = os.path.dirname(filename)
name = os.path.splitext(file)[0]
ext = os.path.splitext(file)[1]
dst = path+"\\"+proj_no_title+'_'+str(date)+'_'+name+'_HD'+ext # manipulate as required
src = filename
dest = Path(dst)
sour = Path(src)
# rename() function will
# rename all the files
# print(sour,dest)
os.rename(src, dst)
rename()
Using the Command Window to rename image files
Using the command window to rename image files can be achieved using the same csv file that was employed for the Bulk Rename Utility. Run the command prompt window within the directory containing the csv file (see above). At the prompt, type the following and then press the enter key. Note that if you make a mistake after you issue the command and press the enter key, you can recall the last line typed by pressing the up-arrow key on the keyboard and then you can correct it.
For /f “tokens=1* delims=,” %A in (import.csv) do ren “%A” “%B”
Where “import.csv” contains the old and new filenames.
A possible disadvantage of using this approach is that the files are renamed in situ. More important is that the approach will not work if the filenames (including paths, i.e. directory names) contain spaces (although these could be replaced by underscores first).
Stage 2: Populating the DAMS Metadata spreadsheets
There are detailed DAMS image metadata templates with descriptions on the required content provided in the Statement of Requirements for contracted work, with examples given for how to enter both video and still image metadata. Template spreadsheets are available to facilitate delivery of the media products for upload into NatureScot’s DAMS.
The DAMS Imagery Metadata spreadsheet should be used by the contractor for inputting the metadata associated with the media files (still image and video); including where video files that have been derived from segmenting the original video files into biotope samples (i.e. the spreadsheet can contain the ‘parent’ original video and ‘child’ video segments, where this is applicable).
The separate Camera Methods spreadsheet should be used to define the camera equipment and methods used for collecting the media (stills and video). The method categories, populated in the ‘image_method_ID’ field, are used to link with the metadata in the DAMS Imagery Metadata spreadsheet.
Example information to enter against each metadata fields
DAMS imagery metadata import spreadsheet extracts - examples
| DAMS Field title | Example still image | Example video |
|---|---|---|
| Title | “98226_2015-06-01_DCF1234.jpg” | “98226_2015-06-01_DCF1234.avi” |
| Description | “Image supporting seabird population count of Gannet using AOS method.” | “Video supporting seabird population count of Gannet using AOS method.” |
| Keywords | “Ailsa Craig; seabird; gannet; monitoring; colony; population; boat; census; cliff” | “Ailsa Craig; seabird; gannet; monitoring; colony; population; boat; census; cliff” |
| Asset_Datetime (date/time image taken) | “2015-06-01 10:01:02” | “2015-06-01 10:01:02” |
| Author | [Nominated Officer] or [NatureScot] “Joe Bloggs” “NatureScot” | [Nominated Officer] or [NatureScot] “Joe Bloggs” “NatureScot” |
| Original_Source | “Marine Ornithology” | “Marine Ornithology” |
| Corporate_Value | “Yes” | “Yes” |
| Designation_Type | “SPA” | “SPA” |
| Protected_Area_Name | “Ailsa Craig SPA” | “Ailsa Craig SPA” |
| ProtectedAreaCodeNS | “8463” | “8463” |
| MarineOrnith_StartDate | “2015-06-01” | “2015-06-01” |
| MarineOrnith_EndDate | “2015-06-30” | “2015-06-30” |
MarineOrnith_SiteCode | “98226” | “98226” |
| MarineOrnith_SiteName | “Ailsa Craig” | “Ailsa Craig” |
| MarineOrnith_MasterSiteName | “Ailsa Craig (Master) [86637]” | “Ailsa Craig (Master) [86637]” |
| MarineOrnith_ProjectID | (SMPSiteCode-StartDate) [xxxxx_yyyy-mm-dd] 98226_2015-06-01 | (SMPSiteCode-StartDate) [xxxxx_yyyy-mm-dd] 98226_2015-06-01 |
| Upload_user_email | Contractor to leave blank NatureScot staff email address | Contractor to leave blank NatureScot staff email address |
| MediaType | [select from drop down] “Photo” | [select from drop down] “Video” |
| OriginalFile | “DCF1234” [only populate if the file name has been renamed / updated from originally used in previous reports / data outputs; otherwise enter the current file name (matches with the Title field)] | “DCF1234” [only populate if the file name has been renamed / updated from originally used in previous reports / data outputs; otherwise enter the current file name (matches with the Title field)] |
Camera method information metadata spreadsheet extract - examples
| Field title | Camera method example |
|---|---|
| imageMethodID | “METHOD_311” |
| imageDeviceCategory | “311:Cameras” |
| imageDeviceManufacturer | “FUJIFILM” “Canon” |
| imageDeviceModel | “FinePixS3Pro” |
| imageDeviceConfiguration | “Still images were recorded to camera by surveyors. An overlay was used to provide station metadata in the recorded still image. Still images were captured opportunistically, targeting the subject of survey. EXIF data embedded in images. The camera height above seabed was controlled by the surveyor holding the camera.” |
| imageFormat | “The camera recorded a signal following the 1080i standard, i.e. with a full HD resolution of 1920×1080 pixels. Images were saved on the internal hard drive using e.g. AVCHD Video (.m2ts) or MP4 Video (.MP4) format which is based on the MPEG-4 AVC/H.264 for image compression.” |
| imagePostProcessing | “The images had contrast enhanced by 30%.” |
| imageCalibrationNotes | “The standard zoom setting used was 100mm.” |
| imageMethodNotes | “Poor quality images were removed from the dataset retained.” |
Linking data between counts and media files
Where images or video data are collected during the survey, these should be linked to the count data using a common unique ID. This ID will link data from the site survey (which may be multiple species counts) and the imagery (which may be multiple images or videos). Therefore, an ID field must be generated and used in both the count data and imagery metadata.
MarineOrnith_ProjectID format: SMP_SiteCode-StartDate [xxxxx_yyyy-mm-dd] e.g. ‘98226_2015-06-01’
The MarineOrnith_ProjectID field should be submitted to the SMP database in the comments field.
Data Quality Assurance
Once the data has been processed into the final format (as outlined below) the contractor is required to undertake Quality Assurance checks and validation of the data on at least 10% of the records.
Once completed the data is sent to the NatureScot project Nominated Officer to undergo further internal quality assurance processes. Any QA queries will be communicated between the Nominated Officer and Contractor. We reserve the right to return data to the contractor where data has been incomplete or inaccurate, if necessary.
Once the data has passed QA checks, a final version will be passed back to the contractor for submission to the Seabird Monitoring Programme.
Data submission to the SMP online database
Once the data has been processed and quality assurance checks have been completed, the relevant fields from the colony count and breeding success/productivity data should be submitted to the SMP Online Database. Data entry requires a user to be registered on the website. This can be done by following the instructions on the login page.
A copy of the finalised dataset submitted to the SMP, along with the additional fields specified in these guidance, should also be supplied to NatureScot with the final report.
