PLEXIS Healthcare Systems, Inc., a progressive software development company in Medford, OR, is looking to hire an enthusiastic and experienced LLM Systems Engineer within our AI Department. The candidate must be local to the Southern Oregon/Rogue Valley area. Please apply at www.plexishealth.com

Position Purpose

The LLM Systems Engineer will focus on the design, implementation, and operation of applied Large Language Model (LLM) systems used in production software. The position centers on how LLMs retrieve information, manage context and memory, apply behavioral guardrails, and operate safely with appropriate human oversight. This role does not focus on training or fine-tuning foundation models, but instead on building reliable, observable, and compliant systems around them.

The AI Special Ops team is a dynamic and experimental group focused on exploring cutting-edge AI capabilities and translating them into real-world, production-ready solutions. The LLM Systems Engineer will collaborate closely with engineers, product leaders, and end users to help ensure AI systems are accurate, safe, and trustworthy.

Essential Functions and Basic Duties

Design, build, and maintain Retrieval-Augmented Generation (RAG) pipelines for LLM-powered features
Ingest, structure, and manage data from PDFs, document repositories, databases, and semi-structured sources
Implement chunking strategies, embeddings, vector search, and re-ranking to improve retrieval precision
Develop and maintain context and memory mechanisms for LLM interactions, including relevance and summarization strategies
Apply behavioral and domain guardrails to guide model outputs and prevent inappropriate or unsafe responses
Support Human-in-the-Loop (HITL) escalation workflows for sensitive or ambiguous scenarios
Capture and incorporate explicit and implicit user feedback to improve system behavior over time
Implement logging and observability for LLM inputs, retrieved context, outputs, and guardrail events
Assist with debugging, incident review, and performance tuning of AI systems
Collaborate with cross-functional team members in an agile development environment
Duties are subject to change as projects and priorities evolve
Required to work a minimum of 3 days per week in the office; remaining days may be worked remotely or in-office

Preferred Skills & Experience

1–3 years of professional software development experience
Experience as a backend or full-stack engineer working with APIs and data pipelines
Hands-on exposure to LLMs and applied AI systems through professional work, side projects, or prototypes
Working understanding of embeddings, vector search, and LLM context window limitations
Familiarity with Retrieval-Augmented Generation (RAG) concepts
Ability to reason about model accuracy, hallucinations, and system limitations
Strong problem-solving skills and comfort working in ambiguous, exploratory problem spaces
Effective communication skills and ability to collaborate across technical and non-technical teams

Bonus Skills & Experience

Experience with LLM tooling or frameworks (e.g., LangChain, LlamaIndex, or similar)
Exposure to regulated or high-trust domains such as healthcare, insurance, or finance
Familiarity with logging, monitoring, or observability tooling
Experience contributing to production systems involving AI or automation

Physical Activities and Requirements of this Position

Ability to access input and retrieves information from a computer to produce typed copy
Capacity to communicate adequately, in person or via telephone, in a manner, which can be understood by those with whom the Client Support Specialist is speaking
Capability of dialing or otherwise using a telephone to place and receive telephone calls
Willingness and capacity to sit or stand for minimum periods of one hour at a time
Capacity to pick up, leaf through and read books and files and other materials
Ability to reach forward, up, down and to the side in order to move equipment up to 40 lbs
Ability to interact cohesively with co-workers, clients, vendors and partner companies
Ability to work in a fast-paced environment, with competing deadlines and changing priorities
Hybrid position requires 3 days a week in the office. The remaining 2 days may be worked in the office or remotely
Willingness and ability to maintain regular attendance for normally scheduled hours, with flexibility as needed

Working Conditions

When in office; ability to tolerate, use, work with, in, or under:

Florescent lighting
Recycled air
VDTs
Semi-enclosed areas
Central heating and air conditioning
Office noise
Occasional moving to new seating areas and buildings.

The key physical requirements for this position include the ability to move freely through an office environment. Use of standard office equipment including PCs, Fax, Copiers, and Phone Systems.

Mental Activities and Requirements of this Position

Reasoning Ability

Possesses and uses good diagnosis and troubleshooting skills.
Able to identify problems, patterns, and think through potential solutions then communicates and/or escalates appropriately.

Mathematics Ability

Basic math skills.

Language Ability

Capacity to communicate adequately, in person or via telephone, in an easily understood manner.

Position Details:

Position Location: Hybrid - 3 Days in Office, 2 Days Remotely or in Office

Position Status: Full-Time/Exempt

Benefits: Competitive Healthcare Package; Medical, Dental, Vision and 401K

Time Off: Competitive Vacation and Sick Time Package

Salary Range: $96,000 - $120,000/annually Depending on Experience

It is the policy of Plexis Healthcare Systems, Inc. to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law

Llm Systems Engineer - Ai Dept.

Job description