{"id":196,"date":"2018-08-29T19:42:25","date_gmt":"2018-08-29T19:42:25","guid":{"rendered":"http:\/\/dsail.csail.mit.edu\/?page_id=196"},"modified":"2018-09-30T14:10:50","modified_gmt":"2018-09-30T14:10:50","slug":"scheduler","status":"publish","type":"page","link":"https:\/\/dsail.csail.mit.edu\/index.php\/scheduler\/","title":{"rendered":"Scheduler"},"content":{"rendered":"<h2>\n\t\tScheduler: An Opportunity for Reinforcement Learning<br \/>\n\t<\/h2>\n<p>Today&#8217;s database systems use simple scheduling policies like first-come-first-serve for their generality and ease of implementation.&nbsp; However, a scheduler customized for a specific workload can perform a variety of optimizations; for example, it can prioritize fast and low-cost queries, select query-specific parallelism thresholds, and order operations in query execution to avoid bottlenecks (e.g., leverage query structure to run slow stages in parallel with other non-dependent stages).&nbsp; Such workload-specific policies are rarely used in practice because they require expert knowledge and take significant effort to devise, implement, and validate.<\/p>\n<p>In DAS we envision a new scheduling system that automatically learns highly efficient scheduling policies tailored to the data and workload.&nbsp; Our system represents a scheduling algorithm as a neural network that takes as input information about the data (e.g., using a CDF model) and the query workload (e.g., using a model trained on previous executions of queries) to make scheduling decisions. We train the scheduling neural network using modern reinforcement learning (RL) techniques to optimize high-level system objectives such as minimal average query completion time.<\/p>\n<p>\t\t\t\t<img loading=\"lazy\" src=\"http:\/\/dsail.csail.mit.edu\/wp-content\/uploads\/2018\/09\/1807.02264.jpg\" alt=\"1807.02264\" itemprop=\"image\" height=\"326\" width=\"251\" title=\"1807.02264\"  \/><\/p>\n<p><strong>Citations<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Scheduler: An Opportunity for Reinforcement Learning Today&#8217;s database systems use simple scheduling policies like first-come-first-serve for their generality and ease of implementation.&nbsp; However, a scheduler customized for a specific workload can perform a variety of optimizations; for example, it can prioritize fast and low-cost queries, select query-specific parallelism thresholds, and order operations in query execution&hellip;<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/pages\/196"}],"collection":[{"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/comments?post=196"}],"version-history":[{"count":13,"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/pages\/196\/revisions"}],"predecessor-version":[{"id":429,"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/pages\/196\/revisions\/429"}],"wp:attachment":[{"href":"https:\/\/dsail.csail.mit.edu\/index.php\/wp-json\/wp\/v2\/media?parent=196"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}