Meeting 6

(May 31, 2024)

Attendees

• Anupam Ghosh
• Shaheem Azmal M MD
• Gaurav Mishra
• Kaushlendra Pratap
• Aaditya Singh

Discussion

Contributor

• Aaditya Singh: Focused on framing the current scheduler by studying the Job Scheduler wiki Documentation.

Current Scheduler Design

The schematic from the documentation provides an architectural overview of the current Job Scheduler:

Architectural Overview:

The design is based on a client-server model, featuring:

1. Scheduler Port: Serves as the communication hub between clients and the scheduler.
2. Main Thread: Responsible for job queuing, allocation, and event management.
3. Worker Threads: Implement specific tasks asynchronously, enhancing throughput and response times.

Key Components:

1. Scheduler: Manages all job scheduling operations and acts as the primary interface for job requests.
2. Asynchronous Workers: Improve processing time by handling tasks in parallel.
3. Logging and Monitoring: Provides integrated systems for real-time monitoring and issue troubleshooting.

Alignment with Documentation

Mentor:

• Gaurav Mishra: Confirmed alignment with the current architecture of the job scheduler.

New Scheduler Design

The revised design focuses on a more efficient multithreaded approach:

Architectural Overview:

1. Main Thread: Coordinates with various components and manages task execution.
2. Worker Thread: Manages interactions between different services to prevent conflicts.
3. Event Queue: Ensures proper handling of system-level events and error logging.

Key Components:

1. Scheduler: Uses a round-robin technique to manage tasks, ensuring efficient CPU time distribution.
2. Queue Storage: Manages tasks dynamically to adapt to workload changes.

Trade-offs

Feature	Old Job Scheduler	New Job Scheduler
Maintainability	Easier	More challenging
Monitoring	Centralized	Decentralized
Resource Management	Static	Dynamic

Effectiveness of New Design

Mentor:

• Gaurav Mishra: Suggests a finite state machine architecture might be more suitable given the complexity, as it facilitates easier tracking and management.

Addressing FOSSology Issue

#2742: Problem with incorrect MIME type recognition for text files.

Mentor:

• Shaheem Azmal M MD: Error found due to the "Ignore SCM files" option affecting MIME type recognition.

Wfx Project Approach

Mentor:

• Kaushlendra Pratap: Expressed interest in holding an internal discussion about the wfx project approach.

Goals for the Week

Contributor:

• Aaditya Singh: Goals include rewriting the existing C code to Go and implementing a queue in Golang.