or our customer, a major financial services organisation we are seeking a Data Quality Engineer for a long term project focussed on assessment of cyber security issues related to large data sets.
CICD, Elasticsearch experience is a must. o Experience in both elastic ingest and logstash pipeline building, Elastic mapping building, ECS, are a must
Data Source Onboarding
Identify, assess and support owners with new data source onboarding into the data ecosystem
Develop and maintain efficient data pipelines for ingesting and processing diverse types of log source types (Platform, Application, Security, etc)
Data Parsing and Transformation
Design and implement data parsing and transformation to ensure data is structured and accessible
Collaborate with Security Analysts and Data owners to understand data requirement
Data Quality Assurance o Ensure proper data quality standards and processes are maintained
Perform data review, monitor data pipelines and resolve any data quality related issue
User Support o Asist internal stakeholders in accessing and using data and searches more effectively
Troubleshoot queries and provide technical support
Content support o Provide Guidance, create and maintain content, visualization, dashboard, reports on the supported platforms
Create and maintain documentation, including configuration guides and standard operating procedures.