From: Boris Brezillon <boris.brezillon@collabora.com>
To: dri-devel@lists.freedesktop.org
Cc: Tomeu Vizoso <tomeu.vizoso@collabora.com>,
Steven Price <steven.price@arm.com>,
Rob Herring <robh+dt@kernel.org>,
Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>,
Boris Brezillon <boris.brezillon@collabora.com>,
Robin Murphy <robin.murphy@arm.com>
Subject: [PATCH v3 01/15] drm/sched: Allow using a dedicated workqueue for the timeout/fault tdr
Date: Fri, 25 Jun 2021 15:33:13 +0200 [thread overview]
Message-ID: <20210625133327.2598825-2-boris.brezillon@collabora.com> (raw)
In-Reply-To: <20210625133327.2598825-1-boris.brezillon@collabora.com>
Mali Midgard/Bifrost GPUs have 3 hardware queues but only a global GPU
reset. This leads to extra complexity when we need to synchronize timeout
works with the reset work. One solution to address that is to have an
ordered workqueue at the driver level that will be used by the different
schedulers to queue their timeout work. Thanks to the serialization
provided by the ordered workqueue we are guaranteed that timeout
handlers are executed sequentially, and can thus easily reset the GPU
from the timeout handler without extra synchronization.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 2 +-
drivers/gpu/drm/etnaviv/etnaviv_sched.c | 3 ++-
drivers/gpu/drm/lima/lima_sched.c | 3 ++-
drivers/gpu/drm/panfrost/panfrost_job.c | 3 ++-
drivers/gpu/drm/scheduler/sched_main.c | 6 +++++-
drivers/gpu/drm/v3d/v3d_sched.c | 10 +++++-----
include/drm/gpu_scheduler.h | 5 ++++-
7 files changed, 21 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 47ea46859618..532636ea20bc 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -488,7 +488,7 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
num_hw_submission, amdgpu_job_hang_limit,
- timeout, sched_score, ring->name);
+ timeout, NULL, sched_score, ring->name);
if (r) {
DRM_ERROR("Failed to create scheduler on ring %s.\n",
ring->name);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
index 19826e504efc..feb6da1b6ceb 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c
@@ -190,7 +190,8 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu)
ret = drm_sched_init(&gpu->sched, &etnaviv_sched_ops,
etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,
- msecs_to_jiffies(500), NULL, dev_name(gpu->dev));
+ msecs_to_jiffies(500), NULL, NULL,
+ dev_name(gpu->dev));
if (ret)
return ret;
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index ecf3267334ff..dba8329937a3 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -508,7 +508,8 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)
INIT_WORK(&pipe->recover_work, lima_sched_recover_work);
return drm_sched_init(&pipe->base, &lima_sched_ops, 1,
- lima_job_hang_limit, msecs_to_jiffies(timeout),
+ lima_job_hang_limit,
+ msecs_to_jiffies(timeout), NULL,
NULL, name);
}
diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c
index 682f2161b999..8ff79fd49577 100644
--- a/drivers/gpu/drm/panfrost/panfrost_job.c
+++ b/drivers/gpu/drm/panfrost/panfrost_job.c
@@ -626,7 +626,8 @@ int panfrost_job_init(struct panfrost_device *pfdev)
ret = drm_sched_init(&js->queue[j].sched,
&panfrost_sched_ops,
- 1, 0, msecs_to_jiffies(JOB_TIMEOUT_MS),
+ 1, 0,
+ msecs_to_jiffies(JOB_TIMEOUT_MS), NULL,
NULL, "pan_js");
if (ret) {
dev_err(pfdev->dev, "Failed to create scheduler: %d.", ret);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index c0a2f8f8d472..a937d0529944 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -837,6 +837,8 @@ static int drm_sched_main(void *param)
* @hw_submission: number of hw submissions that can be in flight
* @hang_limit: number of times to allow a job to hang before dropping it
* @timeout: timeout value in jiffies for the scheduler
+ * @timeout_wq: workqueue to use for timeout work. If NULL, the system_wq is
+ * used
* @score: optional score atomic shared with other schedulers
* @name: name used for debugging
*
@@ -844,7 +846,8 @@ static int drm_sched_main(void *param)
*/
int drm_sched_init(struct drm_gpu_scheduler *sched,
const struct drm_sched_backend_ops *ops,
- unsigned hw_submission, unsigned hang_limit, long timeout,
+ unsigned hw_submission, unsigned hang_limit,
+ long timeout, struct workqueue_struct *timeout_wq,
atomic_t *score, const char *name)
{
int i, ret;
@@ -852,6 +855,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
sched->hw_submission_limit = hw_submission;
sched->name = name;
sched->timeout = timeout;
+ sched->timeout_wq = timeout_wq ? : system_wq;
sched->hang_limit = hang_limit;
sched->score = score ? score : &sched->_score;
for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_COUNT; i++)
diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c
index 8992480c88fa..a39bdd5cfc4f 100644
--- a/drivers/gpu/drm/v3d/v3d_sched.c
+++ b/drivers/gpu/drm/v3d/v3d_sched.c
@@ -402,7 +402,7 @@ v3d_sched_init(struct v3d_dev *v3d)
ret = drm_sched_init(&v3d->queue[V3D_BIN].sched,
&v3d_bin_sched_ops,
hw_jobs_limit, job_hang_limit,
- msecs_to_jiffies(hang_limit_ms),
+ msecs_to_jiffies(hang_limit_ms), NULL,
NULL, "v3d_bin");
if (ret) {
dev_err(v3d->drm.dev, "Failed to create bin scheduler: %d.", ret);
@@ -412,7 +412,7 @@ v3d_sched_init(struct v3d_dev *v3d)
ret = drm_sched_init(&v3d->queue[V3D_RENDER].sched,
&v3d_render_sched_ops,
hw_jobs_limit, job_hang_limit,
- msecs_to_jiffies(hang_limit_ms),
+ msecs_to_jiffies(hang_limit_ms), NULL,
NULL, "v3d_render");
if (ret) {
dev_err(v3d->drm.dev, "Failed to create render scheduler: %d.",
@@ -424,7 +424,7 @@ v3d_sched_init(struct v3d_dev *v3d)
ret = drm_sched_init(&v3d->queue[V3D_TFU].sched,
&v3d_tfu_sched_ops,
hw_jobs_limit, job_hang_limit,
- msecs_to_jiffies(hang_limit_ms),
+ msecs_to_jiffies(hang_limit_ms), NULL,
NULL, "v3d_tfu");
if (ret) {
dev_err(v3d->drm.dev, "Failed to create TFU scheduler: %d.",
@@ -437,7 +437,7 @@ v3d_sched_init(struct v3d_dev *v3d)
ret = drm_sched_init(&v3d->queue[V3D_CSD].sched,
&v3d_csd_sched_ops,
hw_jobs_limit, job_hang_limit,
- msecs_to_jiffies(hang_limit_ms),
+ msecs_to_jiffies(hang_limit_ms), NULL,
NULL, "v3d_csd");
if (ret) {
dev_err(v3d->drm.dev, "Failed to create CSD scheduler: %d.",
@@ -449,7 +449,7 @@ v3d_sched_init(struct v3d_dev *v3d)
ret = drm_sched_init(&v3d->queue[V3D_CACHE_CLEAN].sched,
&v3d_cache_clean_sched_ops,
hw_jobs_limit, job_hang_limit,
- msecs_to_jiffies(hang_limit_ms),
+ msecs_to_jiffies(hang_limit_ms), NULL,
NULL, "v3d_cache_clean");
if (ret) {
dev_err(v3d->drm.dev, "Failed to create CACHE_CLEAN scheduler: %d.",
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 10225a0a35d0..d4cdc906709e 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -269,6 +269,7 @@ struct drm_sched_backend_ops {
* finished.
* @hw_rq_count: the number of jobs currently in the hardware queue.
* @job_id_count: used to assign unique id to the each job.
+ * @timeout_wq: workqueue used to queue @work_tdr
* @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the
* timeout interval is over.
* @thread: the kthread on which the scheduler which run.
@@ -293,6 +294,7 @@ struct drm_gpu_scheduler {
wait_queue_head_t job_scheduled;
atomic_t hw_rq_count;
atomic64_t job_id_count;
+ struct workqueue_struct *timeout_wq;
struct delayed_work work_tdr;
struct task_struct *thread;
struct list_head pending_list;
@@ -306,7 +308,8 @@ struct drm_gpu_scheduler {
int drm_sched_init(struct drm_gpu_scheduler *sched,
const struct drm_sched_backend_ops *ops,
- uint32_t hw_submission, unsigned hang_limit, long timeout,
+ uint32_t hw_submission, unsigned hang_limit,
+ long timeout, struct workqueue_struct *timeout_wq,
atomic_t *score, const char *name);
void drm_sched_fini(struct drm_gpu_scheduler *sched);
--
2.31.1
next prev parent reply other threads:[~2021-06-25 13:33 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-25 13:33 [PATCH v3 00/15] drm/panfrost: Misc improvements Boris Brezillon
2021-06-25 13:33 ` Boris Brezillon [this message]
2021-06-25 15:07 ` [PATCH v3 01/15] drm/sched: Allow using a dedicated workqueue for the timeout/fault tdr Steven Price
2021-06-25 15:18 ` Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 02/15] drm/panfrost: Make ->run_job() return an ERR_PTR() when appropriate Boris Brezillon
2021-06-25 13:45 ` Alyssa Rosenzweig
2021-06-25 13:33 ` [PATCH v3 03/15] drm/panfrost: Get rid of the unused JS_STATUS_EVENT_ACTIVE definition Boris Brezillon
2021-06-25 13:39 ` Alyssa Rosenzweig
2021-06-25 13:33 ` [PATCH v3 04/15] drm/panfrost: Drop the pfdev argument passed to panfrost_exception_name() Boris Brezillon
2021-06-25 13:45 ` Alyssa Rosenzweig
2021-06-25 13:33 ` [PATCH v3 05/15] drm/panfrost: Expose exception types to userspace Boris Brezillon
2021-06-25 13:42 ` Alyssa Rosenzweig
2021-06-25 14:21 ` Boris Brezillon
2021-06-25 15:32 ` Steven Price
2021-06-25 15:40 ` Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 06/15] drm/panfrost: Do the exception -> string translation using a table Boris Brezillon
2021-06-25 13:41 ` Alyssa Rosenzweig
2021-06-25 15:35 ` Steven Price
2021-06-25 13:33 ` [PATCH v3 07/15] drm/panfrost: Expose a helper to trigger a GPU reset Boris Brezillon
2021-06-25 13:46 ` Alyssa Rosenzweig
2021-06-25 13:33 ` [PATCH v3 08/15] drm/panfrost: Use a threaded IRQ for job interrupts Boris Brezillon
2021-06-25 13:47 ` Alyssa Rosenzweig
2021-06-25 14:37 ` Boris Brezillon
2021-06-25 15:40 ` Steven Price
2021-06-25 13:33 ` [PATCH v3 09/15] drm/panfrost: Simplify the reset serialization logic Boris Brezillon
2021-06-25 15:42 ` Steven Price
2021-06-25 16:22 ` Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 10/15] drm/panfrost: Make sure job interrupts are masked before resetting Boris Brezillon
2021-06-25 15:55 ` Steven Price
2021-06-25 16:02 ` Boris Brezillon
2021-06-25 16:11 ` Steven Price
2021-06-25 13:33 ` [PATCH v3 11/15] drm/panfrost: Disable the AS on unhandled page faults Boris Brezillon
2021-06-25 16:10 ` Steven Price
2021-06-25 13:33 ` [PATCH v3 12/15] drm/panfrost: Reset the GPU when the AS_ACTIVE bit is stuck Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 13/15] drm/panfrost: Don't reset the GPU on job faults unless we really have to Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 14/15] drm/panfrost: Kill in-flight jobs on FD close Boris Brezillon
2021-06-25 13:43 ` Lucas Stach
2021-06-25 14:46 ` Boris Brezillon
2021-06-25 13:33 ` [PATCH v3 15/15] drm/panfrost: Queue jobs on the hardware Boris Brezillon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210625133327.2598825-2-boris.brezillon@collabora.com \
--to=boris.brezillon@collabora.com \
--cc=alyssa.rosenzweig@collabora.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=steven.price@arm.com \
--cc=tomeu.vizoso@collabora.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).