IPND (Integrated Public Number Database)

Purpose
This is a publically accessible website for IPND users to access:
  • Diagnostic tools
  • Validators
  • FAQs
  • Other item that may be of assistance
      
Note
This site:
  • Does NOT Store or retain any data provided
  • Does NOT Connect to any IPND system
  • Is a standalone website to provide user asistance re common issues
  • Shares only Material that is currently accessible in the public domain

Upload/Comprehension File Testing and Validation Plus .err File Analysis

In the sections below you will be able to request analysis of 3 distinct types of files commonly encountered/used when applying changes to the IPND system. These three file types are:

  • Upload Files - A file structured to carry the content to be applied to the IPND on behalf of your organisation
  • Error Files (.err) - A response file matching the upload file processed intended to communicate any issues with the records processed
  • Rejected Files - A response file generated to communicate a fatal error with the matching upload file processed

Each section below will provide detailed validation and analysis of the file types indicated to assist you with issues you may encounter.

Validate an Upload/Comprehension File

Purpose - Validate and Analyse an Upload file:
  • Validate a Comprehension file (an example upload file)
  • Validate any actual (unencrypted) Upload file (of less than 1000 records)
  • Upload files are the one intended to be placed in your roor folder for processing
  • Are of the form: IPNDUPAAAAA.0000001 (minus the .asc since it must be unencrypted to be used here)
  • Any errors in the syntax of the header/trailer or data records will be highlighted
       Actions - What you can do
  • Enter the name of upload file below (used in validation process)
  • Paste the unecrypted contents of the upload file below
  • The contents will be analysed and presented for easy viewing
  • No data provided will be stored (content will be discarded when you close the web page)
  • NEW - Locality, State, Postcode validation now added
Upload FileName:
     
Paste the unencrypted Upload/Comprehension file contents here for validation and viewing    
Total records in pasted file: 0

Upload File - Summary of Findings

Header Record

Data Record

Show Data Record No:

Data Summary

Trailer Record

Tips - Construction/Format of the upload file

When you construct an IPND upload file (or comprehension file), note the following:
  • The file name:
    • All letters must be upper case
    • Format is: IPND<TT><XXXXX>.<NNNNNNN>
      • "IPND" is a constant,
      • <TT> refers to the file type which is "UP" for Upload file,>
      • <XXXXX> is the File Source,
      • "." is an extension marker,
      • <NNNNNNN> is a file sequence number with leading 0
      • example: "IPNDUPABCDE.0000001" where "ABCDE" is the file source
      • Do NOT add an extension such as ".txt" or anything else
    • Note that the encrypted file when submitted, will have an extension of ".asc"
  • The file is a simple text file
    • Any text editor can be used to create it
    • Every record is the same fixed length (905 characters)
    • Each record ends with a standard end of line character (e.g. happens when you add a new row in a text editor)
    • Contains standard keyboard characters
  • The structure of the File
    • The first record must be the header record (i.e. starts with "HDR....")
    • The last record must be the trailer record (i.e. starts with "TRL....")
    • The records in between the header and trailer are the data records
    • Refer to the Technical Specification (see section 6.1.2 Customer Record IPND Upload File)
  • How data fields are represented
    • There is a specific record format for header, trailer and data records (see link above)
    • The data in all records must follow the specifications (i.e. each field has a start position and a length)
      • Think of the data as occupying a fixed position within each record. There are NO field delimiters
      • The specifications show a "From" and "To" column position of the field in the record
      • This means the data must start at the "From" column and not exceed the "To" column
      • Fields named "Filler" must contain spaces
      • Example: In the Header record, the field "File Source" has a specification of From=10, To=14
        • ".........ABCDE................." etc (dots shown only to illustrate position of data)
        • Note that the File Source is shown in postions 10 to 14 inclusive within this record (length=5)
        • Where the data content is less than the length of the field, it should be filled with spaces
    • Treat every field as if left aligned and padded with spaces for the length of the field
    • Public numbers should NOT include the country code

Analyse an Upload Error File (.err)

Purpose - Analyse Error files .err:
  • Are files of the form with a suffix .err file
  • Will appear only in your download folder
  • Are of the form: IPNDUPAAAAA.0000001.001.err
  • Purpose is to communicate any issues with the content of the upload file that has been processeed
  • If only a header and trailer record is present, it means there are NO errors
       Actions - What you can do
  • Paste the unecrypted contents of the .err file below
  • The contents will be analysed and presented for easy viewing
Paste the unencrypted .err file contents here for analysis and viewing    
Total records pasted: 0

.err file

Header Record

Error Records

Trailer Record

Tips - understanding the .err response file

About the .err file:
  • The file name:
    • The start of the file name will match the upload file that you submitted
    • It will have a sequence number appended to it:
      • A sequence number like ".001" or ".002"
      • The sequence number increments if the same upload file name is sent multiple times
      • The highest sequence number matches the most recent file sent with that same name
    • The file will have an extension of ".err" when unencrypted
  • The .err file:
    • will be placed in your download folder
    • Is an encrypted file that you must retrieve and then decrypt using your gpg key
    • The File will always contain a Header (HDR...) and Trailer record (TRL...)
    • If the file contains just a header and trailer record, then this signifies there are NO Errors
    • Every record between the Header and trailer records (if any) indicates the presence of a specific error
    • If there is no matching .err file for a file you uploaded, check your "rejected" folder
  • The meaning of the .err file records
    • Every record (excluding the header and trailer records) reports one error
    • Each record shows the public number impacted, its record number in the matching upload file and the error number and error type
    • Errors can be of types W=Warning, S=Soft, H=Hard, F=Fatal
    • The error number indicates the specific error that has been detected
    • The most common errors are Soft errors 50,51,52,53 which indicate that an invalid combination of Suburb, State and postcode was detected
    • Refer to the Australia Post website to find or validate combinations to use
  • Handling errors reported
    • Any Fatal error will cause the entire file to be ignored (not processed)
    • Hard errors will stop just the specific change record from being processed
    • Soft and Warning errors will allow the change record to be applied whenever possible
    • If the file contains Any Fatal errors, then the next upload file must use the same upload sequence number (do not increment it)
    • If the file contains No Fatal errors, then the next upload file must increment the upload sequence number
    • You should attempt to correct all errors in your next upload file

Analyse a Rejected Upload File (.Rnnn)

Purpose - Rejected files .Rnnn file:
  • Are files of the form with a suffix .Rnnn
  • Will appear only in your rejected folder
  • Are empty files and only the suffix code provides the error meaning
       Actions - What you can do
  • Enter the name of the file found in your rejected folder
  • The suffix will be used to display the meaning of the error indicated
Rejected Upload FileName:
   

Upload Error Codes

Purpose - the meaning of error codes
  • After a file is uploaded, errors are communicated via codes
  • Error codes can be of types (F=Fatal, H=Hard, S=Soft, W=Warning)
  • Any Fatal error will cause the entire file to be ignored (not processed)
  • Hard errors will stop just the specific change record from being processed
  • Soft and Warning errors will allow the change record to be applied whenever possible
  • Rejected codes apply only to files that have been placed in the rejected folder (code appended to response file name)
       Further info on Error Types:
  • Error types F,H,S and W
    • These errors can appear in the response file created after a file is uploaded and processed
    • This file is known as the ".err" file and the file name echoes the upload filename
    • The .err file can be found in your "download" folder
    • You must retrieve it and decrypt it to see its contents
    • If the file contains only a header and trailer record, it means no errors where found
    • If there are additional records then each one represents a error for the record No. indicated
    • A record can have multiple errors, each on a new line (one error per line in .err file)
  • Error type R (Rejected)
    • Unlike the above, these errors are indicated in the name of the file produced
    • These files are appear in your "rejected" folder with a name beginning with the name of the uploaded file
    • The error suffix will represent the first reason found for rejecting the file of the form "Rnnn"
    • They are empty files and only the suffix code provides the error meaning
Scroll table or Filter using options:
 
Error No.
Error Type (S,H,W,F,R)
  
Text search

Valid FileSource / ProviderCode / CSP Combinations

Purpose - the meaning of these valid combinations
  • The IPND system maintains a list of the known validate FileSource / ProviderCode / CSP Combinations
  • From 1 Jul 2024 these valid combinations will be enforced as validation within upload files
  • Upload files will generate errors if an invalid combination is used
       Further info on Valid Combinations:
  • Invalid combinations will generate a fatal error number 040
  • These errors will appear in the response file created after a file is uploaded and processed

Check for valid combinations by
entering one or more of the following: 		 
Provider Code
  
FileSource
  
CSP Code
  

Partial input accepted		  

Valid Location / State / Postcode Combinations

Purpose - the meaning of these valid combinations
  • The IPND system maintains a list of the known validate Location / Suburb / Postcode Combinations
  • These valid combinations are enforced as validation within upload files for the Service Address Fields
  • Upload files will generate errors if an invalid combination is used
  • Valid combinations are sourced from Australia Post and are updated periodically
       Further info on Valid Combinations:
  • Invalid combinations will generate a fatal error number depending on the field, i.e 33, 34, 35, 51, 52, 53
  • These errors will appear in the response file created after a file is uploaded and processed

Check for valid combinations by
entering one or more of the following: 		 
Locality (Suburb)
  
State
  
Postcode
  

Partial input accepted		  

IPND Web Portal Help

What is the IPND Web Portal?

The IPND Web Interface (known as the portal) provides services that are already available via the current IPND Core system, however is expanding to cater for more user types (e.g. Carriage Service Providers or CSPs) and to facilitate access to numerous standard reports and repetative scheduling (ordering) of extracts (reports). It is envisaged that the current system will continue to provide these services, viz:

  • Ability to transfer some types of files,
  • Scheduling of extracts,
  • Viewing of Service meta data,
  • Scheduling and download of management reports etc.
Change planned for July 2024, will allow CSP Organisations access to the portal as a new class of users relating to CSP Organisations and more importantly, supports the ability for the CSP Organisation's Users to Download Encrypted Reports directly.

IPND web Portal Help

The IPND Web Portal can only be assessed via vpn access, username and password which must first be established via an onboarding process. However the web page help is readily accessible publically which you can review. Links to various sections of this help are listed below (click to open new tab to view):
  • About the IPND System
  • IPND Help Centre: Home
  • Getting Started on the Web Portal
  • Users and Actors
  • Regulatory Bodies
  • The IPND Manager
  • Data Provider User
  • Data Consumer User
  • Carriage Service Provider User
  • Topics index
  • Glossary of Terms

Frequently Asked Questions (FAQs)

Does my vpn login work for Test and Production use?

Yes, the same vpn credentials are used for production and test

Do I need a unique set of ssh and gpg keys for both test and production?

No, the ssh and gpg keys that you provide for test access will also be used for production access

What are the upload file size contraints?

Upload files are limited to 100,000 records. Files transmitted with more than 100,000 records (excluding header and footer) will be rejected with a file level error of 241F.

An Upload file with this number of rows is approximately 90 Mb in size. This minimises the amount of data that needs to be re-transmitted in the event of a network failure during the file transfer process, reduces the effect on downstream users and preserves the integrity of the IPND application.

It is important that Data Providers do not upload more than 200,000 records in a single day without advising the IPND Manager of their intention to do so.

The IPND will not produce files that contain more than 100,000 records (exclusive of Header and Trailer).

NOTE: This limit for upload files is 10,000 (exclusive of header and trailer) in the usertest/onboarding environment.

What is the naming format for an upload file?

Note that as the initial checking will reject files if they do not match this pattern.

Note that all letters must be upper case.

File Naming format:

  • IPND<TT><XXXXX>.<NNNNNNN><NNNNNNN>
    Where
    • "IPND" is a constant,
    • <TT> refers to the file type which is "UP" for Upload file,
    • <XXXXX> is the File Source,
    • "." is an extension marker,
    • <NNNNNNN> is a file sequence number with leading 0.
  • Any file that fails to comply with the above name structure will be rejected and reported in the Error file.

When should the File Sequence number be incremented?

IPND will maintain a log of loaded Upload File from a Data Provider.

File Sequence Numbers will be sequential and contiguous.

The File Sequence Number of the last successfully loaded Upload File for a File Source will be compared with the File Sequence Number of the current file.
If Last File Sequence Number + 1 does not equal Current File Sequence Number then the file will be rejected and reported in the Error file.

This has the following implications. If a file is rejected because of a Hard Error at the file level, e.g. the Header Record count does not match the actual number of records in the file, then no further files may be processed from that provider until a file arrives with that specific sequence number.
Thus if file f5 is rejected on day 1, file f6 that arrives on day 2 and file f7 that arrives on day 3 will not be loaded until file f5 is corrected and re-transmitted.

File Transfer directory folders explained

Each client user who has been approved and established with access to transfer files to the IPND will have access to a folder structure as follows:
  • Root folder typically named (p_xxxxx where xxxx is your Filesource code). This is where files to be uploaded (processed) are to be placed.
    • download - this is where response files are placed eg .err and extract files requested/produced. It contains a user's encrypted download files that are less than 6 months old
    • rejected - contains files that a user has uploaded, but were rejected for processing by the IPND. Rejected files are held in this directory as zero byte versions of the files for a limited period before being deleted. The extension of the file (.Rnnn) indicates the nature of the fatal error (e.g. naming, encryption errors)
    • archived - contains encrypted files from the download directory that are older than 6 months and younger than 24 months. Files older than 24 months are deleted.
    • received - contains a copy of recently uploaded files. Received files are held in this directory for a limited period before being deleted.

Maintenance Timeframes

A maintenance window of 12:00 (noon) and 14:00 daily will be reserved. Unless there are exceptional circumstances, users will be given two weeks notice of any maintenance that may disrupt the file transfer window.

Question

Answer

Resources

Here is a list of resources that you may find useful:
  • (IPND) Data Users and Data Providers Technical Requirements for IPND
    • Purpose: The technical specifications for data users and data providers when uploading/retrieving files

  • IPND Data Users and Data Providers Access to Internet Interface Service (IIS)
    • Purpose: Describes how to establish a connection to the IPND Internet Interface Service (IIS)

  • IPND Data Industry Guideline
  • Industry Code C555:2017 (IPND)
    • Purpose: Communications Alliance Ltd was formed in 1997 to provide a unified voice for the Australian communications industry and to lead it into the next generation of converging networks, technologies and services.

  • Australia Post Web Site
    • Purpose: To locate valid Suburb,State & Postcode combinations

  • IPND Help desk customer support
    • Purpose: You can raise a request for IPND Customer Support using the form provided

  • Telstra IPND IIS User Login
    • Purpose: Using your assigned username/password, retrieve you vpn credentials (.ovpn file)

  • Login to the IPND Website
    • Purpose: Access to the IPND web portal (You must be connected the vpn first)

XXX

IPND Manager - All General communications, New Users, Procedural Assistance etc, should go to the IPND Manager
Email: send email to ipnd.manager@team.telstra.com 		Technical assistance - For specific issues encountered by existing Data Providers or Data Users
Email: Send an email to ipndhelp@logicaltech.com.au      website: Customer Support Help Desk