- Create and improve processes, tools, workflows, and resilient data architecture to scrape web content.
- Manage data accuracy and quality.
- Identify and rectify any issues with breaks as well as scale scrapers as needed.
- Support downstream data processes including cleaning, normalization and enrichments.
- Experience in developing web scraping solutions and architecture
- Experience developing resilient architecture and quality control and maintenance processes
- Strong web scraping experience (Scrapy preferred, other packages considered)
- Experience with ETL and creating data pipelines
- Familiar with common anti-scraping strategies (dynamic data loading, Reverse JS, IP proxy, cookie pool, authentication code, etc.)
- Basic Linux and Git experience
- Python scrapy library
- AWS/AWS Batch
- Familiarity with data processing tools (pandas, regex, SQL), preferred
- ElasticSearch experience, preferred
Other jobs you may like
Perfect Sense is a visionary technology company that empowers digital possibility through Brightspot®, a content management and distribution engine built for high-volume portfolio, media, and brand publishers.
Do you want to work hard alongside smart and talented product managers, engineers, and designers every day? Do you excel when you are smack in the middle of a challenging project, thrive when things get complex, and yawn when everything is going according to plan? Do you actually want to like your clients and coworkers?
Our DevOps Engineers practice a hybrid of System Administration, Configuration/Release Management, and Systems Integration. At Perfect Sense, it’s not just about supporting software releases, it’s about departmental collaboration to bring success to our clients. This involves participation in company-wide strategic initiatives, diving deeper into complex engineering environments and finding solutions to tough problems.
- Implement and manage multi-server deployments in AWS, Rackspace Cloud and Microsoft Azure
- Collaborate with development teams to deploy new code to production environments
- Maintain and improve monitoring systems and alarms for production environments, using Zabbix, Pingdom, ElasticSearch and Graylog
- Set up back-up mechanisms for production environment and test disaster recovery mechanisms
- Mentor and train junior DevOps engineers
- Manage incident response with other engineers and clients
- Participate in rotating 24/7 on-call schedules with team members
- 3-5 years of experience combining Development and Operational Administration
- Experience as a systems administrator in a Unix environment. (Linux, Ubuntu, Redhat)
- Outstanding communication skills with the ability to work in a client facing role
- Outstanding problem-solving skills in situations with tricky technical situations
- In-depth knowledge of an infrastructure automation framework.
- Proficiency in Python and shell scripting
- Experience setting up and managing Databases – MySQL preferred
- Experience setting up and managing Tomcat, Apache, and/or Solr
- Knowledge and experience of typical build and release management processes, including continuous integration packages such as Travis
- Knowledge and experience of Zabbix and Pingdom
- Knowledge and experience of Elastic Search and Graylog for log management
- Knowledge and experience with Docker containers in cloud environments
Other jobs you may like
About Cigna Digital:
By joining the Digital team, you’ll have a unique opportunity to transform healthcare and make a positive impact on millions of lives.
The Digital Team creates engaging web and mobile experiences that make it easier for consumers to find high quality and affordable healthcare. The Digital team evolved from a tech startup formerly known as Brighter. Brighter was acquired by Cigna in 2017 to lead the digital transformation of its health plans, thus forming Cigna’s Digital team. Our mission is to bring increased transparency and accessibility to healthcare by creating simple consumer driven digital experiences. We’re unique in that we offer the stability and benefits of a large company with the culture of a startup. We’re located just a few blocks from the promenade in Santa Monica, and pride ourselves on having a tight-knit, collaborative team that likes to solve complex problems.
When joining this DevOps Team, you will be bringing Cigna into the cloud, vetting out solutions, building out the infrastructure, generating the tooling, lightly coding when needed and you will be viewed at the “cloud expert”. You will work independently in close partnership with cross-functional teams and management.
System real-time monitoring, metrics and alerts across all applications
Develop deployment and automation tools
Build systems that dynamically scale
Plan deployments for zero down-time
2+ years of *nix experience (or 2+ years of development experience)
Nice to have:
Relational Database experience / basic administration (Postgres, Mysql, etc)
NoSQL experience (DynamoDB, Cassandra, etc)
Search solutions experience (Elasticsearch)
CI/CD experience (Gitlab CI, Github Actions, Jenkins, etc),
Configuration Management (Ansible, etc),
Infrastructure as Code (Terraform, Cloudformation, etc)
Cigna Corporation (NYSE: CI) is a global health service company dedicated to improving the health, well-being and peace of mind of those we serve. We offer an integrated suite of health services through Cigna, Express Scripts, and our affiliates including medical, dental, behavioral health, pharmacy, vision, supplemental benefits, and other related products. Together, with our 74,000 employees worldwide, we aspire to transform health services, making them more affordable and accessible to millions. Through our unmatched expertise, bold action, fresh ideas and an unwavering commitment to patient-centered care, we are a force of health services innovation.
When you work with Cigna, you’ll enjoy meaningful career experiences that enrich people’s lives while working together to make the world a healthier place. What difference will you make? To see our culture in action, search #TeamCigna on Instagram.
Qualified applicants will be considered without regard to race, color, age, disability, sex, childbirth (including pregnancy) or related medical conditions including but not limited to lactation, sexual orientation, gender identity or expression, veteran or military status, religion, national origin, ancestry, marital or familial status, genetic information, status with regard to public assistance, citizenship status or any other characteristic protected by applicable equal employment opportunity laws.
If you require an accommodation based on your physical or mental disability please email: [email protected] Do not email [email protected] for an update on your application or to provide your resume as you will not receive a response.