feat:新加入定时任务功能
This commit is contained in:
parent
7debee74e0
commit
9e59170a4b
95
README.md
95
README.md
@ -19,13 +19,14 @@
|
||||
|
||||
Kinit 是一套全部开源的快速开发平台,毫无保留给个人及企业免费使用。
|
||||
|
||||
- 后端采用 Python 语言现代、快速(高性能) [FastAPI](https://fastapi.tiangolo.com/zh/) 异步框架 + [SQLAlchemy](https://www.sqlalchemy.org/) 异步操作 [MySQL](https://www.mysql.com/) 数据库。
|
||||
- PC端采用 [vue-element-plus-admin](https://gitee.com/kailong110120130/vue-element-plus-admin) 、[Vue3](https://cn.vuejs.org/guide/introduction.html)、[Element Plus](https://element-plus.gitee.io/zh-CN/guide/design.html)、[TypeScript](https://www.tslang.cn/)等主流技术开发。
|
||||
- 移动端采用 [uni-app](https://uniapp.dcloud.net.cn/component/),[Vue2](https://v2.cn.vuejs.org/v2/guide/),[uView 2](https://www.uviewui.com/components/intro.html)为主要技术开发
|
||||
- 新加入 [Typer](https://typer.tiangolo.com/) 命令行应用,简单化数据初始化,数据表模型迁移。
|
||||
- 后端采用 Python 语言现代、快速(高性能) [FastAPI](https://fastapi.tiangolo.com/zh/) 异步框架 + [SQLAlchemy](https://www.sqlalchemy.org/) 异步操作 [MySQL](https://www.mysql.com/) 数据库;
|
||||
- PC端采用 [vue-element-plus-admin](https://gitee.com/kailong110120130/vue-element-plus-admin) 、[Vue3](https://cn.vuejs.org/guide/introduction.html)、[Element Plus](https://element-plus.gitee.io/zh-CN/guide/design.html)、[TypeScript](https://www.tslang.cn/)等主流技术开发;
|
||||
- 移动端采用 [uni-app](https://uniapp.dcloud.net.cn/component/),[Vue2](https://v2.cn.vuejs.org/v2/guide/),[uView 2](https://www.uviewui.com/components/intro.html)为主要技术开发;
|
||||
- 后端加入 [Typer](https://typer.tiangolo.com/) 命令行应用,简单化数据初始化,数据表模型迁移等操作;
|
||||
- 新加入定时任务功能,采用 [APScheduler](https://github.com/agronholm/apscheduler) 定时任务框架 + [Redis](https://redis.io/) 消息队列 + [MongoDB](https://www.mongodb.com/) 持久存储;
|
||||
- 权限认证使用[(哈希)密码和 JWT Bearer 令牌的 OAuth2](https://fastapi.tiangolo.com/zh/tutorial/security/oauth2-jwt/),支持多终端认证系统。
|
||||
- 支持加载动态权限菜单,多方式轻松权限控制,按钮级别权限控制。
|
||||
- 已加入常见的`Redis`、`MYSQL`、`MongoDB`数据库异步操作。
|
||||
- 已加入常见的 [MySQL](https://www.mysql.com/) + [MongoDB](https://www.mongodb.com/) + [Redis](https://redis.io/) 数据库异步操作。
|
||||
- 开箱即用的中后台解决方案,可以用来作为新项目的启动模版,也可用于学习参考。并且时刻关注着最新技术动向,尽可能的第一时间更新。
|
||||
- 与 [vue-element-plus-admin](https://gitee.com/kailong110120130/vue-element-plus-admin) 前端框架时刻保持同步更新。
|
||||
|
||||
@ -119,6 +120,8 @@ github地址:https://github.com/vvandk/kinit
|
||||
|
||||
- [x] 命令行操作:新加入 `Typer` 命令行应用,简单化数据初始化,数据表模型迁移。
|
||||
|
||||
- [x] 定时任务:在线操作(添加、修改、删除)任务调度包含查看任务执行结果日志。
|
||||
|
||||
## 移动端内置功能
|
||||
|
||||
- [x] 登录认证:支持用户使用手机号+密码方式登录,微信手机号一键登录方式。
|
||||
@ -133,10 +136,9 @@ github地址:https://github.com/vvandk/kinit
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] 考虑支持多机部署方案,如果接口使用多机,那么用户是否支持统一认证
|
||||
- [ ] **自动化编排服务:使用docker-compose部署项目**
|
||||
- [ ] **定时任务:定时执行数据库备份**
|
||||
- [ ] **可视化低代码表单:接入低代码表单,https://vform666.com/vform3.html?from=element_plus**
|
||||
- [ ] 多租户方案
|
||||
- [ ] 自动化编排服务:使用docker-compose部署项目
|
||||
- [ ] 可视化低代码表单:接入低代码表单,[vform3](https://vform666.com/vform3.html?from=element_plus)
|
||||
|
||||
## 前序准备
|
||||
|
||||
@ -145,7 +147,8 @@ github地址:https://github.com/vvandk/kinit
|
||||
- [Python3](https://www.python.org/downloads/windows/):熟悉 python3 基础语法
|
||||
- [FastAPI](https://fastapi.tiangolo.com/zh/) - 熟悉后台接口 Web 框架.
|
||||
- [Typer](https://typer.tiangolo.com/) - 熟悉命令行工具的使用
|
||||
- [MySQL](https://www.mysql.com/) 和 [MongoDB](https://www.mongodb.com/) - 熟悉数据存储数据库
|
||||
- [MySQL](https://www.mysql.com/) 和 [MongoDB](https://www.mongodb.com/) 和 [Redis](https://redis.io/) - 熟悉数据存储数据库
|
||||
- [iP查询接口文档](https://user.ip138.com/ip/doc):IP查询第三方服务,有1000次的免费次数
|
||||
|
||||
### PC端
|
||||
|
||||
@ -157,30 +160,23 @@ github地址:https://github.com/vvandk/kinit
|
||||
- [Vue-Router-Next](https://gitee.com/link?target=https%3A%2F%2Fnext.router.vuejs.org%2F) - 熟悉 vue-router 基本使用
|
||||
- [Element-Plus](https://gitee.com/link?target=https%3A%2F%2Felement-plus.org%2F) - element-plus 基本使用
|
||||
- [Mock.js](https://gitee.com/link?target=https%3A%2F%2Fgithub.com%2Fnuysoft%2FMock) - mockjs 基本语法
|
||||
- [vue3-json-viewer](https://gitee.com/isfive/vue3-json-viewer):简单易用的json内容展示组件,适配vue3和vite。
|
||||
- [SortableJS/vue.draggable.next](https://github.com/SortableJS/vue.draggable.next):Vue 组件 (Vue.js 3.0) 允许拖放和与视图模型数组同步。
|
||||
- [高德地图API (amap.com)](https://lbs.amap.com/api/jsapi-v2/guide/webcli/map-vue1):地图 JSAPI 2.0 是高德开放平台免费提供的第四代 Web 地图渲染引擎。
|
||||
|
||||
### 移动端
|
||||
|
||||
- [uni-app](https://uniapp.dcloud.net.cn/component/) - 熟悉 uni-app 基本语法
|
||||
- [Vue2](https://v2.cn.vuejs.org/v2/guide/) - 熟悉 Vue 基础语法
|
||||
- [uView UI 2](https://www.uviewui.com/components/intro.html):uView UI 组件的基本使用
|
||||
|
||||
### 依赖包
|
||||
|
||||
#### PC端
|
||||
|
||||
- [vue3-json-viewer](https://gitee.com/isfive/vue3-json-viewer):简单易用的json内容展示组件,适配vue3和vite。
|
||||
- [vue3-slide-verify](https://github.com/monoplasty/vue3-slide-verify):滑块验证码插件 vue3 + typescript。
|
||||
- [SortableJS/vue.draggable.next](https://github.com/SortableJS/vue.draggable.next):Vue 组件 (Vue.js 3.0) 允许拖放和与视图模型数组同步。
|
||||
- [高德地图API (amap.com)](https://lbs.amap.com/api/jsapi-v2/guide/webcli/map-vue1):地图 JSAPI 2.0 是高德开放平台免费提供的第四代 Web 地图渲染引擎, 以 WebGL 为主要绘图手段,本着“更轻、更快、更易用”的服务原则,广泛采用了各种前沿技术,交互体验、视觉体验大幅提升,同时提供了众多新增能力和特性。
|
||||
|
||||
#### 移动端
|
||||
|
||||
- [uni-read-pages](https://github.com/SilurianYang/uni-read-pages) :自动读取 `pages.json` 所有配置。
|
||||
- [uni-simple-router](https://hhyang.cn/v2/start/quickstart.html) :在uni-app中使用vue-router的方式进行跳转路由,路由拦截。
|
||||
|
||||
#### 后端
|
||||
### 定时任务
|
||||
|
||||
- [iP查询接口文档](https://user.ip138.com/ip/doc):IP查询第三方服务,有1000次的免费次数
|
||||
- [Python3](https://www.python.org/downloads/windows/) -熟悉 python3 基础语法
|
||||
- [APScheduler](https://github.com/agronholm/apscheduler) - 熟悉定时任务框架
|
||||
- [MongoDB](https://www.mongodb.com/) 和 [Redis](https://redis.io/) - 熟悉数据存储数据库
|
||||
|
||||
## 安装和使用
|
||||
|
||||
@ -323,6 +319,12 @@ Redis (推荐使用最新稳定版)
|
||||
# 微信小程序配置
|
||||
wx_server_app_id
|
||||
wx_server_app_secret
|
||||
|
||||
# 邮箱配置
|
||||
email_access
|
||||
email_password
|
||||
email_server
|
||||
email_port
|
||||
```
|
||||
|
||||
6. 启动
|
||||
@ -354,6 +356,51 @@ pnpm run dev
|
||||
pnpm run build:pro
|
||||
```
|
||||
|
||||
### 定时任务
|
||||
|
||||
1. 安装依赖
|
||||
|
||||
```
|
||||
# 安装依赖库
|
||||
pip3 install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# 第三方源:
|
||||
1. 阿里源: https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
2. 修改项目数据库配置信息
|
||||
|
||||
在 `application/config` 目录中
|
||||
|
||||
- development.py:开发环境
|
||||
|
||||
- production.py:生产环境
|
||||
|
||||
```python
|
||||
"""
|
||||
MongoDB 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
MONGO_DB_NAME = "数据库名称"
|
||||
MONGO_DB_URL = f"mongodb://用户名:密码@地址:端口/?authSource={MONGO_DB_NAME}"
|
||||
|
||||
|
||||
"""
|
||||
Redis 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
REDIS_DB_URL = "redis://:密码@地址:端口/数据库名称"
|
||||
```
|
||||
|
||||
3. 启动
|
||||
|
||||
```
|
||||
python3 main.py
|
||||
```
|
||||
|
||||
|
||||
### 访问项目
|
||||
|
||||
- 访问地址:http://localhost:5000 (默认为此地址,如有修改请按照配置文件)
|
||||
|
BIN
images/new/1688392266702.jpg
Normal file
BIN
images/new/1688392266702.jpg
Normal file
Binary file not shown.
After Width: | Height: | Size: 83 KiB |
33
kinit-admin/src/api/vadmin/system/task.ts
Normal file
33
kinit-admin/src/api/vadmin/system/task.ts
Normal file
@ -0,0 +1,33 @@
|
||||
import request from '@/config/axios'
|
||||
|
||||
export const getTaskListApi = (params: any): Promise<IResponse> => {
|
||||
return request.get({ url: '/vadmin/system/tasks', params })
|
||||
}
|
||||
|
||||
export const addTaskListApi = (data: any): Promise<IResponse> => {
|
||||
return request.post({ url: '/vadmin/system/tasks', data })
|
||||
}
|
||||
|
||||
export const delTaskListApi = (dataId: string): Promise<IResponse> => {
|
||||
return request.delete({ url: `/vadmin/system/tasks?_id=${dataId}` })
|
||||
}
|
||||
|
||||
export const putTaskListApi = (dataId: string, data: any): Promise<IResponse> => {
|
||||
return request.put({ url: `/vadmin/system/tasks?_id=${dataId}`, data })
|
||||
}
|
||||
|
||||
export const getTaskApi = (dataId: string): Promise<IResponse> => {
|
||||
return request.get({ url: `/vadmin/system/task?_id=${dataId}` })
|
||||
}
|
||||
|
||||
export const getTaskGroupOptionsApi = (): Promise<IResponse> => {
|
||||
return request.get({ url: '/vadmin/system/task/group/options' })
|
||||
}
|
||||
|
||||
export const getTaskRecordListApi = (params: any): Promise<IResponse> => {
|
||||
return request.get({ url: '/vadmin/system/task/records', params })
|
||||
}
|
||||
|
||||
export const runOnceTaskApi = (dataId: string): Promise<IResponse> => {
|
||||
return request.post({ url: `/vadmin/system/task?_id=${dataId}` })
|
||||
}
|
@ -14,7 +14,7 @@ defineProps({
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<ElCard :class="[prefixCls]" shadow="never">
|
||||
<ElCard :class="[prefixCls]" shadow="never" class="!border-0">
|
||||
<template v-if="title" #header>
|
||||
<div class="flex items-center">
|
||||
<span class="text-16px font-700">{{ title }}</span>
|
||||
|
@ -6,3 +6,9 @@
|
||||
width: 100% !important;
|
||||
}
|
||||
|
||||
// 解决element-plus 中的日期组件宽度无法设置100%的问题
|
||||
.el-date-editor .el-input__wrapper {
|
||||
width: 100% !important;
|
||||
}
|
||||
|
||||
|
||||
|
@ -64,4 +64,6 @@
|
||||
--app-footer-height: 50px;
|
||||
|
||||
--transition-time-02: 0.2s;
|
||||
|
||||
--app-content-bg-color-new: #ffffff;
|
||||
}
|
||||
|
@ -8,8 +8,8 @@
|
||||
export interface DictDetail {
|
||||
label: string
|
||||
value: string
|
||||
disabled: boolean
|
||||
is_default: boolean
|
||||
disabled?: boolean
|
||||
is_default?: boolean
|
||||
}
|
||||
export const selectDictLabel = (datas: DictDetail[], value: string) => {
|
||||
if (!value) {
|
||||
|
@ -0,0 +1,20 @@
|
||||
<script setup lang="ts">
|
||||
import { PropType } from 'vue'
|
||||
import { Descriptions } from '@/components/Descriptions'
|
||||
import { DescriptionsSchema } from '@/types/descriptions'
|
||||
|
||||
defineProps({
|
||||
currentRow: {
|
||||
type: Object as PropType<Nullable<any>>,
|
||||
default: () => null
|
||||
},
|
||||
detailSchema: {
|
||||
type: Array as PropType<DescriptionsSchema[]>,
|
||||
default: () => []
|
||||
}
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<Descriptions :schema="detailSchema" :data="currentRow || {}" />
|
||||
</template>
|
@ -0,0 +1,115 @@
|
||||
import { FormSchema } from '@/types/form'
|
||||
import { TableColumn } from '@/types/table'
|
||||
import { reactive } from 'vue'
|
||||
|
||||
export const columns = reactive<TableColumn[]>([
|
||||
{
|
||||
field: 'job_id',
|
||||
label: '任务编号',
|
||||
show: true,
|
||||
disabled: true,
|
||||
width: '240px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'name',
|
||||
label: '任务名称',
|
||||
show: true,
|
||||
disabled: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'group',
|
||||
label: '任务分组',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'job_class',
|
||||
label: '调用目标',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'exec_strategy',
|
||||
label: '执行策略',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'expression',
|
||||
label: '表达式',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'start_time',
|
||||
label: '开始执行时间',
|
||||
show: false,
|
||||
width: '200px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'end_time',
|
||||
label: '执行完成时间',
|
||||
width: '200px',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'process_time',
|
||||
label: '耗时(秒)',
|
||||
width: '110px',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'retval',
|
||||
label: '任务返回值',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'exception',
|
||||
label: '异常信息',
|
||||
show: false,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'traceback',
|
||||
label: '堆栈跟踪',
|
||||
show: false,
|
||||
width: '100px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'action',
|
||||
width: '100px',
|
||||
label: '操作',
|
||||
show: true,
|
||||
disabled: true,
|
||||
span: 24
|
||||
}
|
||||
])
|
||||
|
||||
export const searchSchema = reactive<FormSchema[]>([
|
||||
{
|
||||
field: 'job_id',
|
||||
label: '任务编号',
|
||||
component: 'Input',
|
||||
componentProps: {
|
||||
clearable: true,
|
||||
style: {
|
||||
width: '240px'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'name',
|
||||
label: '任务名称',
|
||||
component: 'Input',
|
||||
componentProps: {
|
||||
clearable: true
|
||||
}
|
||||
}
|
||||
])
|
138
kinit-admin/src/views/vadmin/system/record/task/index.vue
Normal file
138
kinit-admin/src/views/vadmin/system/record/task/index.vue
Normal file
@ -0,0 +1,138 @@
|
||||
<script lang="ts">
|
||||
export default {
|
||||
name: 'SystemRecordTask'
|
||||
}
|
||||
</script>
|
||||
|
||||
<script setup lang="ts">
|
||||
import { ContentWrap } from '@/components/ContentWrap'
|
||||
import { Table } from '@/components/Table'
|
||||
import { getTaskRecordListApi } from '@/api/vadmin/system/task'
|
||||
import { useTable } from '@/hooks/web/useTable'
|
||||
import { columns, searchSchema } from './components/task.data'
|
||||
import { ref, watch, nextTick } from 'vue'
|
||||
import { ElButton, ElRow } from 'element-plus'
|
||||
import { RightToolbar } from '@/components/RightToolbar'
|
||||
import { Dialog } from '@/components/Dialog'
|
||||
import Detail from './components/Detail.vue'
|
||||
import { Search } from '@/components/Search'
|
||||
import { useCache } from '@/hooks/web/useCache'
|
||||
import { useRouter } from 'vue-router'
|
||||
import { DictDetail, selectDictLabel } from '@/utils/dict'
|
||||
import { useDictStore } from '@/store/modules/dict'
|
||||
import { FormSetPropsType } from '@/types/form'
|
||||
|
||||
const { wsCache } = useCache()
|
||||
const { currentRoute } = useRouter()
|
||||
|
||||
const job_id = currentRoute.value.query.job_id
|
||||
|
||||
const { register, elTableRef, tableObject, methods } = useTable({
|
||||
getListApi: getTaskRecordListApi,
|
||||
response: {
|
||||
data: 'data',
|
||||
count: 'count'
|
||||
}
|
||||
})
|
||||
tableObject.params = { job_id: job_id }
|
||||
|
||||
const dialogVisible = ref(false)
|
||||
const dialogTitle = ref('')
|
||||
const execStrategyOptions = ref<DictDetail[]>([])
|
||||
const searchSetSchemaList = ref([] as FormSetPropsType[])
|
||||
|
||||
if (typeof job_id === 'string') {
|
||||
searchSetSchemaList.value.push({
|
||||
field: 'job_id',
|
||||
path: 'value',
|
||||
value: job_id
|
||||
})
|
||||
}
|
||||
|
||||
const getOptions = async () => {
|
||||
const dictStore = useDictStore()
|
||||
const dictOptions = await dictStore.getDictObj(['vadmin_system_task_exec_strategy'])
|
||||
execStrategyOptions.value = dictOptions.vadmin_system_task_exec_strategy
|
||||
}
|
||||
|
||||
getOptions()
|
||||
|
||||
const view = (row: any) => {
|
||||
dialogTitle.value = '操作记录'
|
||||
tableObject.currentRow = row
|
||||
dialogVisible.value = true
|
||||
}
|
||||
|
||||
const { getList, setSearchParams } = methods
|
||||
|
||||
getList()
|
||||
|
||||
const tableSize = ref('default')
|
||||
|
||||
watch(tableSize, (val) => {
|
||||
tableSize.value = val
|
||||
})
|
||||
|
||||
const route = useRouter()
|
||||
const cacheTableHeadersKey = route.currentRoute.value.fullPath
|
||||
|
||||
watch(
|
||||
columns,
|
||||
async (val) => {
|
||||
wsCache.set(cacheTableHeadersKey, JSON.stringify(val))
|
||||
await nextTick()
|
||||
elTableRef.value?.doLayout()
|
||||
},
|
||||
{
|
||||
deep: true
|
||||
}
|
||||
)
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<ContentWrap>
|
||||
<Search
|
||||
:schema="searchSchema"
|
||||
:setSchemaList="searchSetSchemaList"
|
||||
@search="setSearchParams"
|
||||
@reset="setSearchParams"
|
||||
/>
|
||||
|
||||
<div class="mb-8px flex justify-between">
|
||||
<ElRow />
|
||||
<RightToolbar
|
||||
@get-list="getList"
|
||||
v-model:table-size="tableSize"
|
||||
v-model:columns="columns"
|
||||
:cache-table-headers-key="cacheTableHeadersKey"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Table
|
||||
v-model:limit="tableObject.limit"
|
||||
v-model:page="tableObject.page"
|
||||
:columns="columns"
|
||||
:data="tableObject.tableData"
|
||||
:loading="tableObject.loading"
|
||||
:selection="false"
|
||||
:size="tableSize"
|
||||
:border="true"
|
||||
:pagination="{
|
||||
total: tableObject.count
|
||||
}"
|
||||
@register="register"
|
||||
>
|
||||
<template #exec_strategy="{ row }">
|
||||
{{ selectDictLabel(execStrategyOptions, row.exec_strategy) }}
|
||||
</template>
|
||||
|
||||
<template #action="{ row }">
|
||||
<ElButton type="primary" link size="small" @click="view(row)"> 详情 </ElButton>
|
||||
</template>
|
||||
</Table>
|
||||
|
||||
<Dialog v-model="dialogVisible" :title="dialogTitle" width="900px">
|
||||
<Detail :detail-schema="columns" :current-row="tableObject.currentRow" />
|
||||
</Dialog>
|
||||
</ContentWrap>
|
||||
</template>
|
112
kinit-admin/src/views/vadmin/system/task/components/Write.vue
Normal file
112
kinit-admin/src/views/vadmin/system/task/components/Write.vue
Normal file
@ -0,0 +1,112 @@
|
||||
<script setup lang="ts">
|
||||
import { Form } from '@/components/Form'
|
||||
import { useForm } from '@/hooks/web/useForm'
|
||||
import { PropType, reactive, watch } from 'vue'
|
||||
import { useValidator } from '@/hooks/web/useValidator'
|
||||
import { schema } from './task.data'
|
||||
import { DictDetail } from '@/utils/dict'
|
||||
import { ElRadioGroup, ElRadio } from 'element-plus'
|
||||
|
||||
const { required } = useValidator()
|
||||
|
||||
const props = defineProps({
|
||||
currentRow: {
|
||||
type: Object as PropType<Nullable<any>>,
|
||||
default: () => null
|
||||
},
|
||||
execStrategyOptions: {
|
||||
type: Object as PropType<DictDetail[]>,
|
||||
default: () => null
|
||||
},
|
||||
taskGroupOptions: {
|
||||
type: Object as PropType<DictDetail[]>,
|
||||
default: () => null
|
||||
}
|
||||
})
|
||||
|
||||
const rules = reactive({
|
||||
name: [required()],
|
||||
exec_strategy: [required()],
|
||||
expression: [{ required: true, message: '请填写表达式', trigger: 'blur' }],
|
||||
job_class: [required()]
|
||||
})
|
||||
|
||||
const { register, methods, elFormRef } = useForm({
|
||||
schema: schema
|
||||
})
|
||||
|
||||
watch(
|
||||
() => props.currentRow,
|
||||
(currentRow) => {
|
||||
if (!currentRow) return
|
||||
const { setValues } = methods
|
||||
setValues(currentRow)
|
||||
},
|
||||
{
|
||||
deep: true,
|
||||
immediate: true
|
||||
}
|
||||
)
|
||||
|
||||
watch(
|
||||
() => props.execStrategyOptions,
|
||||
(execStrategyOptions) => {
|
||||
if (!execStrategyOptions) return
|
||||
const { setSchema } = methods
|
||||
setSchema([
|
||||
{
|
||||
field: 'exec_strategy',
|
||||
path: 'componentProps.options',
|
||||
value: execStrategyOptions
|
||||
}
|
||||
])
|
||||
},
|
||||
{
|
||||
deep: true,
|
||||
immediate: true
|
||||
}
|
||||
)
|
||||
|
||||
watch(
|
||||
() => props.taskGroupOptions,
|
||||
(taskGroupOptions) => {
|
||||
if (!taskGroupOptions) return
|
||||
const { setSchema } = methods
|
||||
setSchema([
|
||||
{
|
||||
field: 'group',
|
||||
path: 'componentProps.options',
|
||||
value: taskGroupOptions
|
||||
}
|
||||
])
|
||||
},
|
||||
{
|
||||
deep: true,
|
||||
immediate: true
|
||||
}
|
||||
)
|
||||
|
||||
const handleChange = (form) => {
|
||||
form['start_date'] = null
|
||||
form['end_date'] = null
|
||||
form['expression'] = null
|
||||
elFormRef.value?.clearValidate('expression')
|
||||
}
|
||||
|
||||
defineExpose({
|
||||
elFormRef,
|
||||
getFormData: methods.getFormData
|
||||
})
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<Form :rules="rules" @register="register">
|
||||
<template #exec_strategy="form">
|
||||
<ElRadioGroup v-model="form['exec_strategy']" @change="handleChange(form)">
|
||||
<ElRadio v-for="(item, $index) in execStrategyOptions" :key="$index" :label="item.value">{{
|
||||
item.label
|
||||
}}</ElRadio>
|
||||
</ElRadioGroup>
|
||||
</template>
|
||||
</Form>
|
||||
</template>
|
326
kinit-admin/src/views/vadmin/system/task/components/task.data.ts
Normal file
326
kinit-admin/src/views/vadmin/system/task/components/task.data.ts
Normal file
@ -0,0 +1,326 @@
|
||||
import { FormSchema } from '@/types/form'
|
||||
import { TableColumn } from '@/types/table'
|
||||
import { reactive } from 'vue'
|
||||
|
||||
export const columns = reactive<TableColumn[]>([
|
||||
{
|
||||
field: '_id',
|
||||
label: '任务编号',
|
||||
show: true,
|
||||
disabled: true,
|
||||
width: '240px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'name',
|
||||
label: '任务名称',
|
||||
width: '200px',
|
||||
show: true,
|
||||
disabled: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'group',
|
||||
label: '任务分组',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'job_class',
|
||||
label: '调用目标',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'exec_strategy',
|
||||
label: '执行策略',
|
||||
component: 'Radio',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'expression',
|
||||
label: '表达式',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'is_active',
|
||||
label: '任务状态',
|
||||
show: true,
|
||||
width: '100px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'last_run_datetime',
|
||||
label: '最近一次执行时间',
|
||||
show: true,
|
||||
width: '200px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'remark',
|
||||
label: '任务备注',
|
||||
show: true,
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'create_datetime',
|
||||
label: '创建时间',
|
||||
show: true,
|
||||
width: '200px',
|
||||
span: 24
|
||||
},
|
||||
{
|
||||
field: 'action',
|
||||
label: '操作',
|
||||
show: true,
|
||||
disabled: false,
|
||||
width: '240px',
|
||||
span: 24
|
||||
}
|
||||
])
|
||||
|
||||
export const schema = reactive<FormSchema[]>([
|
||||
{
|
||||
field: 'name',
|
||||
label: '任务名称',
|
||||
component: 'Input',
|
||||
colProps: {
|
||||
span: 12
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'group',
|
||||
label: '任务分组',
|
||||
colProps: {
|
||||
span: 12
|
||||
},
|
||||
component: 'Select',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
allowCreate: true,
|
||||
filterable: true,
|
||||
defaultFirstOption: true,
|
||||
placeholder: '请选择任务分组,支持直接输入添加'
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'job_class',
|
||||
label: '调用目标',
|
||||
component: 'Input',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
placeholder:
|
||||
'调用示例:test.main.Test("kinit", 1314, True);参数仅支持字符串,整数,浮点数,布尔类型。'
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'exec_strategy',
|
||||
label: '执行策略',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
component: 'Radio',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
}
|
||||
},
|
||||
value: 'interval'
|
||||
},
|
||||
{
|
||||
field: 'expression',
|
||||
label: '表达式',
|
||||
component: 'Input',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
placeholder:
|
||||
'interval 表达式,五位,分别为:秒 分 时 天 周,例如:10 * * * * 表示每隔 10 秒执行一次任务。'
|
||||
},
|
||||
ifshow: (values) => values.exec_strategy === 'interval'
|
||||
},
|
||||
{
|
||||
field: 'expression',
|
||||
label: '表达式',
|
||||
component: 'Input',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
placeholder: 'cron 表达式,六位或七位,分别表示秒、分钟、小时、天、月、星期几、年'
|
||||
},
|
||||
ifshow: (values) => values.exec_strategy === 'cron'
|
||||
},
|
||||
{
|
||||
field: 'expression',
|
||||
label: '执行时间',
|
||||
component: 'DatePicker',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
type: 'datetime',
|
||||
format: 'YYYY-MM-DD HH:mm:ss',
|
||||
valueFormat: 'YYYY-MM-DD HH:mm:ss'
|
||||
},
|
||||
ifshow: (values) => values.exec_strategy === 'date'
|
||||
},
|
||||
{
|
||||
field: 'start_date',
|
||||
label: '开始时间',
|
||||
colProps: {
|
||||
span: 12
|
||||
},
|
||||
component: 'DatePicker',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
type: 'datetime',
|
||||
format: 'YYYY-MM-DD HH:mm:ss',
|
||||
valueFormat: 'YYYY-MM-DD HH:mm:ss'
|
||||
},
|
||||
ifshow: (values) => values.exec_strategy !== 'date'
|
||||
},
|
||||
{
|
||||
field: 'end_date',
|
||||
label: '结束时间',
|
||||
colProps: {
|
||||
span: 12
|
||||
},
|
||||
component: 'DatePicker',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
type: 'datetime',
|
||||
format: 'YYYY-MM-DD HH:mm:ss',
|
||||
valueFormat: 'YYYY-MM-DD HH:mm:ss'
|
||||
},
|
||||
ifshow: (values) => values.exec_strategy !== 'date'
|
||||
},
|
||||
{
|
||||
field: 'is_active',
|
||||
label: '任务状态',
|
||||
colProps: {
|
||||
span: 8
|
||||
},
|
||||
component: 'Radio',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
options: [
|
||||
{
|
||||
label: '正常',
|
||||
value: true
|
||||
},
|
||||
{
|
||||
label: '停用',
|
||||
value: false
|
||||
}
|
||||
]
|
||||
},
|
||||
value: true
|
||||
},
|
||||
{
|
||||
field: '',
|
||||
label: '',
|
||||
colProps: {
|
||||
span: 16
|
||||
},
|
||||
component: 'Text',
|
||||
value:
|
||||
'创建或更新任务完成后,如果任务状态与设置的不符,请尝试刷新数据或查看调度日志,任务状态可能会有延迟(几秒)。'
|
||||
},
|
||||
{
|
||||
field: 'remark',
|
||||
label: '备注说明',
|
||||
component: 'Input',
|
||||
colProps: {
|
||||
span: 24
|
||||
},
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '100%'
|
||||
},
|
||||
maxlength: '1000',
|
||||
showWordLimit: true,
|
||||
type: 'textarea',
|
||||
rows: '3'
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'active',
|
||||
label: ' ',
|
||||
colProps: {
|
||||
span: 24
|
||||
}
|
||||
}
|
||||
])
|
||||
|
||||
export const searchSchema = reactive<FormSchema[]>([
|
||||
{
|
||||
field: 'name',
|
||||
label: '任务名称',
|
||||
component: 'Input',
|
||||
componentProps: {
|
||||
clearable: true,
|
||||
style: {
|
||||
width: '214px'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
field: '_id',
|
||||
label: '任务编号',
|
||||
component: 'Input',
|
||||
componentProps: {
|
||||
clearable: true,
|
||||
style: {
|
||||
width: '214px'
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
field: 'group',
|
||||
label: '任务分组',
|
||||
component: 'Select',
|
||||
componentProps: {
|
||||
style: {
|
||||
width: '214px'
|
||||
},
|
||||
options: []
|
||||
}
|
||||
}
|
||||
])
|
289
kinit-admin/src/views/vadmin/system/task/index.vue
Normal file
289
kinit-admin/src/views/vadmin/system/task/index.vue
Normal file
@ -0,0 +1,289 @@
|
||||
<script lang="ts">
|
||||
export default {
|
||||
name: 'HelpTask'
|
||||
}
|
||||
</script>
|
||||
|
||||
<script setup lang="ts">
|
||||
import { ContentWrap } from '@/components/ContentWrap'
|
||||
import { Table } from '@/components/Table'
|
||||
import {
|
||||
getTaskListApi,
|
||||
addTaskListApi,
|
||||
delTaskListApi,
|
||||
putTaskListApi,
|
||||
getTaskApi,
|
||||
getTaskGroupOptionsApi,
|
||||
runOnceTaskApi
|
||||
} from '@/api/vadmin/system/task'
|
||||
import { useTable } from '@/hooks/web/useTable'
|
||||
import { columns, searchSchema } from './components/task.data'
|
||||
import { ref, watch, nextTick, unref } from 'vue'
|
||||
import { ElRow, ElCol, ElButton, ElSwitch, ElMessageBox, ElMessage } from 'element-plus'
|
||||
import { RightToolbar } from '@/components/RightToolbar'
|
||||
import { useDictStore } from '@/store/modules/dict'
|
||||
import { selectDictLabel, DictDetail } from '@/utils/dict'
|
||||
import { FormSetPropsType } from '@/types/form'
|
||||
import { Search } from '@/components/Search'
|
||||
import { useI18n } from '@/hooks/web/useI18n'
|
||||
import { Dialog } from '@/components/Dialog'
|
||||
import { useCache } from '@/hooks/web/useCache'
|
||||
import { useRouter } from 'vue-router'
|
||||
import Write from './components/Write.vue'
|
||||
|
||||
const { wsCache } = useCache()
|
||||
const { t } = useI18n()
|
||||
const { push } = useRouter()
|
||||
|
||||
const { register, elTableRef, tableObject, methods } = useTable({
|
||||
getListApi: getTaskListApi,
|
||||
delListApi: delTaskListApi,
|
||||
response: {
|
||||
data: 'data',
|
||||
count: 'count'
|
||||
}
|
||||
})
|
||||
|
||||
const { getList, setSearchParams } = methods
|
||||
|
||||
const tableSize = ref('default')
|
||||
|
||||
watch(tableSize, (val) => {
|
||||
tableSize.value = val
|
||||
})
|
||||
|
||||
const route = useRouter()
|
||||
const cacheTableHeadersKey = route.currentRoute.value.fullPath
|
||||
|
||||
watch(
|
||||
columns,
|
||||
async (val) => {
|
||||
wsCache.set(cacheTableHeadersKey, JSON.stringify(val))
|
||||
await nextTick()
|
||||
elTableRef.value?.doLayout()
|
||||
},
|
||||
{
|
||||
deep: true
|
||||
}
|
||||
)
|
||||
|
||||
const dialogVisible = ref(false)
|
||||
const dialogTitle = ref('')
|
||||
const loading = ref(false)
|
||||
const actionType = ref('')
|
||||
const execStrategyOptions = ref<DictDetail[]>([])
|
||||
const taskGroupOptions = ref<DictDetail[]>([])
|
||||
const searchSetSchemaList = ref([] as FormSetPropsType[])
|
||||
|
||||
const getOptions = async () => {
|
||||
const dictStore = useDictStore()
|
||||
const dictOptions = await dictStore.getDictObj(['vadmin_system_task_exec_strategy'])
|
||||
execStrategyOptions.value = dictOptions.vadmin_system_task_exec_strategy
|
||||
const res = await getTaskGroupOptionsApi()
|
||||
taskGroupOptions.value = res.data.map((item) => {
|
||||
return {
|
||||
label: item.value,
|
||||
value: item.value
|
||||
}
|
||||
})
|
||||
searchSetSchemaList.value.push({
|
||||
field: 'group',
|
||||
path: 'componentProps.options',
|
||||
value: taskGroupOptions.value
|
||||
})
|
||||
}
|
||||
|
||||
getOptions()
|
||||
|
||||
// 跳转到调度日志页面
|
||||
const toRecord = (row: any) => {
|
||||
if (row) {
|
||||
console.log(row)
|
||||
push(`/system/record/task?job_id=${row._id}`)
|
||||
} else {
|
||||
push(`/system/record/task`)
|
||||
}
|
||||
}
|
||||
|
||||
// 添加定时任务事件
|
||||
const addAction = async () => {
|
||||
dialogTitle.value = '添加定时任务'
|
||||
tableObject.currentRow = null
|
||||
dialogVisible.value = true
|
||||
actionType.value = 'add'
|
||||
}
|
||||
|
||||
// 编辑事件
|
||||
const updateAction = async (row: any) => {
|
||||
const res = await getTaskApi(row._id)
|
||||
if (res) {
|
||||
dialogTitle.value = '编辑定时任务'
|
||||
tableObject.currentRow = res.data
|
||||
dialogVisible.value = true
|
||||
actionType.value = 'edit'
|
||||
}
|
||||
}
|
||||
|
||||
// 删除事件
|
||||
const delData = async (row: any) => {
|
||||
const { delListApi } = methods
|
||||
await delListApi(false, row._id)
|
||||
}
|
||||
|
||||
// 执行一次任务
|
||||
const runOnceTask = async (row: any) => {
|
||||
ElMessageBox.confirm('是否确认立即执行一次任务', t('common.delWarning'), {
|
||||
confirmButtonText: t('common.delOk'),
|
||||
cancelButtonText: t('common.delCancel'),
|
||||
type: 'warning'
|
||||
}).then(async () => {
|
||||
const res = await runOnceTaskApi(row._id)
|
||||
if (res) {
|
||||
if (res.data > 0) {
|
||||
ElMessage.success('任务成功被消费者接收!')
|
||||
} else {
|
||||
ElMessage.error('执行失败,未有消费者接收任务,请检查定时任务程序状态!')
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
const writeRef = ref<ComponentRef<typeof Write>>()
|
||||
|
||||
// 提交事件
|
||||
const save = async () => {
|
||||
const write = unref(writeRef)
|
||||
await write?.elFormRef?.validate(async (isValid) => {
|
||||
if (isValid) {
|
||||
loading.value = true
|
||||
let data = await write?.getFormData()
|
||||
try {
|
||||
const res = ref({} as any)
|
||||
if (data?.exec_strategy === 'date') {
|
||||
data.start_date = null
|
||||
data.end_date = null
|
||||
}
|
||||
if (actionType.value === 'add') {
|
||||
res.value = await addTaskListApi(data)
|
||||
if (res.value) {
|
||||
dialogVisible.value = false
|
||||
const result = res.value.data
|
||||
if (result.is_active) {
|
||||
if (result.subscribe_number > 0) {
|
||||
ElMessage.success('创建成功,任务成功被消费者接收!')
|
||||
} else {
|
||||
ElMessage.info('创建成功,未有消费者接收任务,请检查定时任务程序状态!')
|
||||
}
|
||||
} else {
|
||||
ElMessage.success('创建成功!')
|
||||
}
|
||||
getList()
|
||||
}
|
||||
} else if (actionType.value === 'edit') {
|
||||
res.value = await putTaskListApi(data?._id, data)
|
||||
if (res.value) {
|
||||
dialogVisible.value = false
|
||||
const result = res.value.data
|
||||
if (result.is_active) {
|
||||
if (result.subscribe_number > 0) {
|
||||
ElMessage.success('更新成功,任务已重新被消费者接收!')
|
||||
} else {
|
||||
ElMessage.info('更新成功,未有消费者接收任务,请检查定时任务程序状态!')
|
||||
}
|
||||
} else {
|
||||
ElMessage.success('更新成功!')
|
||||
}
|
||||
getList()
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
loading.value = false
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// 生成 cron 表达式
|
||||
const generateCronExpression = () => {
|
||||
ElMessage.info('下一个版本更新')
|
||||
}
|
||||
|
||||
getList()
|
||||
</script>
|
||||
|
||||
<template>
|
||||
<ContentWrap>
|
||||
<Search
|
||||
:schema="searchSchema"
|
||||
:setSchemaList="searchSetSchemaList"
|
||||
@search="setSearchParams"
|
||||
@reset="setSearchParams"
|
||||
/>
|
||||
|
||||
<div class="mb-8px flex justify-between">
|
||||
<ElRow>
|
||||
<ElCol :span="1.5">
|
||||
<ElButton type="primary" @click="addAction">添加定时任务</ElButton>
|
||||
<ElButton type="primary" @click="toRecord(null)">调度日志</ElButton>
|
||||
<ElButton type="primary" @click="generateCronExpression">生成 Cron 表达式</ElButton>
|
||||
</ElCol>
|
||||
</ElRow>
|
||||
<RightToolbar
|
||||
@get-list="getList"
|
||||
v-model:table-size="tableSize"
|
||||
v-model:columns="columns"
|
||||
:cache-table-headers-key="cacheTableHeadersKey"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Table
|
||||
v-model:limit="tableObject.limit"
|
||||
v-model:page="tableObject.page"
|
||||
:columns="columns"
|
||||
:data="tableObject.tableData"
|
||||
:loading="tableObject.loading"
|
||||
:selection="false"
|
||||
:size="tableSize"
|
||||
:border="true"
|
||||
:pagination="{
|
||||
total: tableObject.count
|
||||
}"
|
||||
@register="register"
|
||||
>
|
||||
<template #is_active="{ row }">
|
||||
<ElSwitch :value="row.is_active" size="small" disabled />
|
||||
</template>
|
||||
|
||||
<template #exec_strategy="{ row }">
|
||||
{{ selectDictLabel(execStrategyOptions, row.exec_strategy) }}
|
||||
</template>
|
||||
|
||||
<template #action="{ row }">
|
||||
<ElButton type="primary" link size="small" @click="updateAction(row)">
|
||||
{{ t('exampleDemo.edit') }}
|
||||
</ElButton>
|
||||
<ElButton type="primary" link size="small" @click="toRecord(row)"> 调度日志 </ElButton>
|
||||
<ElButton type="primary" link size="small" @click="runOnceTask(row)"> 执行一次 </ElButton>
|
||||
<ElButton type="danger" link size="small" @click="delData(row)">
|
||||
{{ t('exampleDemo.del') }}
|
||||
</ElButton>
|
||||
</template>
|
||||
</Table>
|
||||
|
||||
<Dialog v-model="dialogVisible" :title="dialogTitle" width="800px">
|
||||
<Write
|
||||
ref="writeRef"
|
||||
:current-row="tableObject.currentRow"
|
||||
:exec-strategy-options="execStrategyOptions"
|
||||
:task-group-options="taskGroupOptions"
|
||||
/>
|
||||
|
||||
<template #footer>
|
||||
<ElButton type="primary" :loading="loading" @click="save">
|
||||
{{ t('exampleDemo.save') }}
|
||||
</ElButton>
|
||||
<ElButton @click="dialogVisible = false">{{ t('dialogDemo.close') }}</ElButton>
|
||||
</template>
|
||||
</Dialog>
|
||||
</ContentWrap>
|
||||
</template>
|
@ -11,7 +11,7 @@ from fastapi.security import OAuth2PasswordBearer
|
||||
"""
|
||||
系统版本
|
||||
"""
|
||||
VERSION = "1.8.4"
|
||||
VERSION = "1.9.0"
|
||||
|
||||
"""安全警告: 不要在生产中打开调试运行!"""
|
||||
DEBUG = True
|
||||
@ -55,7 +55,7 @@ ACCESS_TOKEN_EXPIRE_MINUTES = 1440
|
||||
"""refresh_token 过期时间,用于刷新token使用,两天"""
|
||||
REFRESH_TOKEN_EXPIRE_MINUTES = 1440 * 2
|
||||
"""access_token 缓存时间,用于刷新token使用,30分钟"""
|
||||
ACCESS_TOKEN_CACHE_MINUTES = 60 * 2
|
||||
ACCESS_TOKEN_CACHE_MINUTES = 30
|
||||
|
||||
"""
|
||||
挂载临时文件目录,并添加路由访问,此路由不会在接口文档中显示
|
||||
@ -134,3 +134,9 @@ MIDDLEWARES = [
|
||||
"core.middleware.register_demo_env_middleware" if DEMO else None,
|
||||
"core.middleware.register_jwt_refresh_middleware"
|
||||
]
|
||||
|
||||
"""
|
||||
定时任务配置
|
||||
"""
|
||||
# 发布/订阅通道,与定时任务程序相互关联,请勿随意更改
|
||||
SUBSCRIBE = 'kinit_queue'
|
||||
|
@ -8,7 +8,6 @@
|
||||
from aioredis import Redis
|
||||
from fastapi import APIRouter, Depends, Body, UploadFile, Request
|
||||
from sqlalchemy.orm import joinedload
|
||||
|
||||
from core.database import redis_getter
|
||||
from utils.response import SuccessResponse, ErrorResponse
|
||||
from . import schemas, crud, models
|
||||
|
@ -7,13 +7,14 @@
|
||||
# @desc : 数据库 增删改查操作
|
||||
import random
|
||||
from typing import List
|
||||
|
||||
# sqlalchemy 查询操作:https://segmentfault.com/a/1190000016767008
|
||||
# sqlalchemy 关联查询:https://www.jianshu.com/p/dfad7c08c57a
|
||||
# sqlalchemy 关联查询详细:https://blog.csdn.net/u012324798/article/details/103940527
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from . import models, schemas
|
||||
from core.crud import DalBase
|
||||
from core.mongo_manage import MongoManage
|
||||
|
||||
|
||||
class LoginRecordDal(DalBase):
|
||||
@ -72,3 +73,9 @@ class SMSSendRecordDal(DalBase):
|
||||
|
||||
def __init__(self, db: AsyncSession):
|
||||
super(SMSSendRecordDal, self).__init__(db, models.VadminSMSSendRecord, schemas.SMSSendRecordSimpleOut)
|
||||
|
||||
|
||||
class OperationRecordDal(MongoManage):
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase):
|
||||
super(OperationRecordDal, self).__init__(db, "operation_record", schemas.OperationRecordSimpleOut)
|
||||
|
@ -1,3 +1,3 @@
|
||||
from .login import LoginRecord, LoginRecordSimpleOut
|
||||
from .sms import SMSSendRecord, SMSSendRecordSimpleOut
|
||||
from .operation import OpertionRecord, OpertionRecordSimpleOut
|
||||
from .operation import OperationRecord, OperationRecordSimpleOut
|
||||
|
@ -11,9 +11,10 @@
|
||||
|
||||
from typing import Optional, List
|
||||
from pydantic import BaseModel
|
||||
from core.data_types import DatetimeStr, ObjectIdStr
|
||||
|
||||
|
||||
class OpertionRecord(BaseModel):
|
||||
class OperationRecord(BaseModel):
|
||||
telephone: Optional[str] = None
|
||||
user_id: Optional[str] = None
|
||||
user_name: Optional[str] = None
|
||||
@ -29,10 +30,10 @@ class OpertionRecord(BaseModel):
|
||||
tags: Optional[List[str]] = None
|
||||
process_time: Optional[str] = None
|
||||
params: Optional[str] = None
|
||||
create_datetime: Optional[str] = None
|
||||
|
||||
|
||||
class OpertionRecordSimpleOut(OpertionRecord):
|
||||
class OperationRecordSimpleOut(OperationRecord):
|
||||
create_datetime: DatetimeStr
|
||||
|
||||
class Config:
|
||||
orm_mode = True
|
||||
|
@ -6,12 +6,14 @@
|
||||
# @desc : 主要接口文件
|
||||
|
||||
from fastapi import APIRouter, Depends
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
|
||||
from utils.response import SuccessResponse
|
||||
from . import crud, schemas
|
||||
from apps.vadmin.auth.utils.current import AllUserAuth
|
||||
from apps.vadmin.auth.utils.validation.auth import Auth
|
||||
from core.mongo import get_database, DatabaseManage
|
||||
from .params import LoginParams, OperationParams, SMSParams
|
||||
from core.database import mongo_getter
|
||||
|
||||
app = APIRouter()
|
||||
|
||||
@ -29,11 +31,11 @@ async def get_record_login(p: LoginParams = Depends(), auth: Auth = Depends(AllU
|
||||
@app.get("/operations", summary="获取操作日志列表")
|
||||
async def get_record_operation(
|
||||
p: OperationParams = Depends(),
|
||||
db: DatabaseManage = Depends(get_database),
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
count = await db.get_count("operation_record", **p.to_count())
|
||||
datas = await db.get_datas("operation_record", v_schema=schemas.OpertionRecordSimpleOut, **p.dict())
|
||||
count = await crud.OperationRecordDal(db).get_count(**p.to_count())
|
||||
datas = await crud.OperationRecordDal(db).get_datas(**p.dict())
|
||||
return SuccessResponse(datas, count=count)
|
||||
|
||||
|
||||
|
@ -11,16 +11,22 @@
|
||||
# sqlalchemy 关联查询详细:https://blog.csdn.net/u012324798/article/details/103940527
|
||||
import json
|
||||
import os
|
||||
from typing import List, Union
|
||||
|
||||
from enum import Enum
|
||||
from typing import List, Union, Any
|
||||
from aioredis import Redis
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
from pymongo.results import InsertOneResult, UpdateResult
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import joinedload
|
||||
from application.settings import STATIC_ROOT
|
||||
from application.settings import STATIC_ROOT, SUBSCRIBE
|
||||
from core.mongo_manage import MongoManage
|
||||
from utils.file.file_manage import FileManage
|
||||
from . import models, schemas
|
||||
from core.crud import DalBase
|
||||
from core.exception import CustomException
|
||||
from utils import status
|
||||
|
||||
|
||||
class DictTypeDal(DalBase):
|
||||
@ -167,3 +173,325 @@ class SettingsTabDal(DalBase):
|
||||
}
|
||||
for tab in datas
|
||||
}
|
||||
|
||||
|
||||
class TaskDal(MongoManage):
|
||||
|
||||
class JobOperation(Enum):
|
||||
add = "add_job"
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase):
|
||||
super(TaskDal, self).__init__(db, "vadmin_system_task", schemas.TaskSimpleOut)
|
||||
|
||||
async def get_task(
|
||||
self,
|
||||
_id: str = None,
|
||||
v_return_none: bool = False,
|
||||
v_schema: Any = None,
|
||||
**kwargs
|
||||
) -> dict | None:
|
||||
"""
|
||||
获取单个数据,默认使用 ID 查询,否则使用关键词查询
|
||||
|
||||
包括临时字段 last_run_datetime,is_active
|
||||
is_active: 只有在 scheduler_task_jobs 任务运行表中存在相同 _id 才表示任务添加成功,任务状态才为 True
|
||||
last_run_datetime: 在 scheduler_task_record 中获取该任务最近一次执行完成的时间
|
||||
|
||||
:param _id: 数据 ID
|
||||
:param v_return_none: 是否返回空 None,否则抛出异常,默认抛出异常
|
||||
:param v_schema: 指定使用的序列化对象
|
||||
"""
|
||||
if _id:
|
||||
kwargs["_id"] = ("ObjectId", _id)
|
||||
|
||||
params = self.filter_condition(**kwargs)
|
||||
pipeline = [
|
||||
{
|
||||
'$addFields': {
|
||||
'str_id': {'$toString': '$_id'}
|
||||
}
|
||||
},
|
||||
{
|
||||
'$lookup': {
|
||||
'from': 'scheduler_task_jobs',
|
||||
'localField': 'str_id',
|
||||
'foreignField': '_id',
|
||||
'as': 'matched_jobs'
|
||||
}
|
||||
},
|
||||
{
|
||||
'$lookup': {
|
||||
'from': 'scheduler_task_record',
|
||||
'localField': 'str_id',
|
||||
'foreignField': 'job_id',
|
||||
'as': 'matched_records'
|
||||
}
|
||||
},
|
||||
{
|
||||
'$addFields': {
|
||||
'is_active': {
|
||||
'$cond': {
|
||||
'if': {'$ne': ['$matched_jobs', []]},
|
||||
'then': True,
|
||||
'else': False
|
||||
}
|
||||
},
|
||||
'last_run_datetime': {
|
||||
'$ifNull': [
|
||||
{'$arrayElemAt': ['$matched_records.create_datetime', -1]},
|
||||
None
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
'$project': {
|
||||
'matched_records': 0,
|
||||
'matched_jobs': 0
|
||||
}
|
||||
},
|
||||
{
|
||||
'$match': params
|
||||
},
|
||||
{
|
||||
'$facet': {
|
||||
'documents': [
|
||||
{'$limit': 1},
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
# 执行聚合查询
|
||||
cursor = self.collection.aggregate(pipeline)
|
||||
result = await cursor.to_list(length=None)
|
||||
data = result[0]['documents']
|
||||
if not data and v_return_none:
|
||||
return None
|
||||
elif not data:
|
||||
raise CustomException("未查找到对应数据", code=status.HTTP_404_NOT_FOUND)
|
||||
data = data[0]
|
||||
if data and v_schema:
|
||||
return jsonable_encoder(v_schema(**data))
|
||||
return data
|
||||
|
||||
async def get_tasks(
|
||||
self,
|
||||
page: int = 1,
|
||||
limit: int = 10,
|
||||
v_schema: Any = None,
|
||||
v_order: str = None,
|
||||
v_order_field: str = None,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
获取任务信息列表
|
||||
|
||||
添加了两个临时字段
|
||||
is_active: 只有在 scheduler_task_jobs 任务运行表中存在相同 _id 才表示任务添加成功,任务状态才为 True
|
||||
last_run_datetime: 在 scheduler_task_record 中获取该任务最近一次执行完成的时间
|
||||
"""
|
||||
v_order_field = v_order_field if v_order_field else 'create_datetime'
|
||||
v_order = -1 if v_order in self.ORDER_FIELD else 1
|
||||
params = self.filter_condition(**kwargs)
|
||||
pipeline = [
|
||||
{
|
||||
'$addFields': {
|
||||
'str_id': {'$toString': '$_id'}
|
||||
}
|
||||
},
|
||||
{
|
||||
'$lookup': {
|
||||
'from': 'scheduler_task_jobs',
|
||||
'localField': 'str_id',
|
||||
'foreignField': '_id',
|
||||
'as': 'matched_jobs'
|
||||
}
|
||||
},
|
||||
{
|
||||
'$lookup': {
|
||||
'from': 'scheduler_task_record',
|
||||
'localField': 'str_id',
|
||||
'foreignField': 'job_id',
|
||||
'as': 'matched_records'
|
||||
}
|
||||
},
|
||||
{
|
||||
'$addFields': {
|
||||
'is_active': {
|
||||
'$cond': {
|
||||
'if': {'$ne': ['$matched_jobs', []]},
|
||||
'then': True,
|
||||
'else': False
|
||||
}
|
||||
},
|
||||
'last_run_datetime': {
|
||||
'$ifNull': [
|
||||
{'$arrayElemAt': ['$matched_records.create_datetime', -1]},
|
||||
None
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
'$project': {
|
||||
'matched_records': 0,
|
||||
'matched_jobs': 0
|
||||
}
|
||||
},
|
||||
{
|
||||
'$match': params
|
||||
},
|
||||
{
|
||||
'$facet': {
|
||||
'documents': [
|
||||
{'$sort': {v_order_field: v_order}},
|
||||
{'$limit': limit},
|
||||
{'$skip': (page - 1) * limit}
|
||||
],
|
||||
'count': [{'$count': 'total'}]
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
# 执行聚合查询
|
||||
cursor = self.collection.aggregate(pipeline)
|
||||
result = await cursor.to_list(length=None)
|
||||
datas = result[0]['documents']
|
||||
count = result[0]['count'][0]['total'] if result[0]['count'] else 0
|
||||
if count == 0:
|
||||
return [], 0
|
||||
elif v_schema:
|
||||
datas = [jsonable_encoder(v_schema(**data)) for data in datas]
|
||||
elif self.schema:
|
||||
datas = [jsonable_encoder(self.schema(**data)) for data in datas]
|
||||
return datas, count
|
||||
|
||||
async def add_task(self, rd: Redis, data: dict) -> int:
|
||||
"""
|
||||
添加任务到消息队列
|
||||
|
||||
使用消息无保留策略:无保留是指当发送者向某个频道发送消息时,如果没有订阅该频道的调用方,就直接将该消息丢弃。
|
||||
|
||||
:params rd: redis 对象
|
||||
:params data: 行数据字典
|
||||
:return: 接收到消息的订阅者数量。
|
||||
"""
|
||||
exec_strategy = data.get("exec_strategy")
|
||||
job_params = {
|
||||
"name": data.get("_id"),
|
||||
"job_class": data.get("job_class"),
|
||||
"expression": data.get("expression")
|
||||
}
|
||||
if exec_strategy == "interval" or exec_strategy == "cron":
|
||||
job_params["start_date"] = data.get("start_date")
|
||||
job_params["end_date"] = data.get("end_date")
|
||||
message = {
|
||||
"operation": self.JobOperation.add.value,
|
||||
"task": {
|
||||
"exec_strategy": data.get("exec_strategy"),
|
||||
"job_params": job_params
|
||||
}
|
||||
}
|
||||
return await rd.publish(SUBSCRIBE, json.dumps(message).encode('utf-8'))
|
||||
|
||||
async def create_task(self, rd: Redis, data: schemas.Task) -> dict:
|
||||
"""
|
||||
创建任务
|
||||
"""
|
||||
data_dict = data.dict()
|
||||
is_active = data_dict.pop('is_active')
|
||||
insert_result = await super().create_data(data_dict)
|
||||
obj = await self.get_task(insert_result.inserted_id, v_schema=schemas.TaskSimpleOut)
|
||||
|
||||
# 如果分组不存在则新增分组
|
||||
group = await TaskGroupDal(self.db).get_data(value=data.group, v_return_none=True)
|
||||
if not group:
|
||||
await TaskGroupDal(self.db).create_data({"value": data.group})
|
||||
|
||||
result = {
|
||||
"subscribe_number": 0,
|
||||
"is_active": is_active
|
||||
}
|
||||
|
||||
if is_active:
|
||||
# 创建任务成功后, 如果任务状态为 True,则向消息队列中发送任务
|
||||
result['subscribe_number'] = await self.add_task(rd, obj)
|
||||
return result
|
||||
|
||||
async def put_task(self, rd: Redis, _id: str, data: schemas.Task) -> dict:
|
||||
"""
|
||||
更新任务
|
||||
"""
|
||||
data_dict = data.dict()
|
||||
is_active = data_dict.pop('is_active')
|
||||
await super(TaskDal, self).put_data(_id, data)
|
||||
obj: dict = await self.get_task(_id, v_schema=schemas.TaskSimpleOut)
|
||||
|
||||
# 如果分组不存在则新增分组
|
||||
group = await TaskGroupDal(self.db).get_data(value=data.group, v_return_none=True)
|
||||
if not group:
|
||||
await TaskGroupDal(self.db).create_data({"value": data.group})
|
||||
|
||||
try:
|
||||
# 删除正在运行中的 Job
|
||||
await SchedulerTaskJobsDal(self.db).delete_data(_id)
|
||||
except CustomException as e:
|
||||
pass
|
||||
|
||||
result = {
|
||||
"subscribe_number": 0,
|
||||
"is_active": is_active
|
||||
}
|
||||
|
||||
if is_active:
|
||||
# 更新任务成功后, 如果任务状态为 True,则向消息队列中发送任务
|
||||
result['subscribe_number'] = await self.add_task(rd, obj)
|
||||
return result
|
||||
|
||||
async def delete_task(self, _id: str) -> bool:
|
||||
"""
|
||||
删除任务
|
||||
"""
|
||||
result = await super(TaskDal, self).delete_data(_id)
|
||||
|
||||
try:
|
||||
# 删除正在运行中的 Job
|
||||
await SchedulerTaskJobsDal(self.db).delete_data(_id)
|
||||
except CustomException as e:
|
||||
pass
|
||||
return result
|
||||
|
||||
async def run_once_task(self, rd: Redis, _id: str) -> int:
|
||||
"""
|
||||
执行一次任务
|
||||
"""
|
||||
obj: dict = await self.get_data(_id, v_schema=schemas.TaskSimpleOut)
|
||||
message = {
|
||||
"operation": self.JobOperation.add.value,
|
||||
"task": {
|
||||
"exec_strategy": "once",
|
||||
"job_params": {
|
||||
"name": obj.get("_id"),
|
||||
"job_class": obj.get("job_class")
|
||||
}
|
||||
}
|
||||
}
|
||||
return await rd.publish(SUBSCRIBE, json.dumps(message).encode('utf-8'))
|
||||
|
||||
|
||||
class TaskGroupDal(MongoManage):
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase):
|
||||
super(TaskGroupDal, self).__init__(db, "vadmin_system_task_group")
|
||||
|
||||
|
||||
class TaskRecordDal(MongoManage):
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase):
|
||||
super(TaskRecordDal, self).__init__(db, "scheduler_task_record")
|
||||
|
||||
|
||||
class SchedulerTaskJobsDal(MongoManage):
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase):
|
||||
super(SchedulerTaskJobsDal, self).__init__(db, "scheduler_task_jobs", is_object_id=False)
|
||||
|
@ -1,2 +1,3 @@
|
||||
from .dict_type import DictTypeParams
|
||||
from .dict_detail import DictDetailParams
|
||||
from .task import TaskParams
|
||||
|
30
kinit-api/apps/vadmin/system/params/task.py
Normal file
30
kinit-api/apps/vadmin/system/params/task.py
Normal file
@ -0,0 +1,30 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/25 14:50
|
||||
# @File : task.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
from fastapi import Depends
|
||||
from core.dependencies import Paging, QueryParams
|
||||
|
||||
|
||||
class TaskParams(QueryParams):
|
||||
"""
|
||||
列表分页
|
||||
"""
|
||||
def __init__(self, name: str = None, _id: str = None, group: str = None, params: Paging = Depends()):
|
||||
super().__init__(params)
|
||||
self.name = ("like", name)
|
||||
self.group = group
|
||||
self._id = ("ObjectId", _id)
|
||||
|
||||
|
||||
class TaskRecordParams(QueryParams):
|
||||
"""
|
||||
列表分页
|
||||
"""
|
||||
def __init__(self, job_id: str = None, name: str = None, params: Paging = Depends()):
|
||||
super().__init__(params)
|
||||
self.job_id = ("like", job_id)
|
||||
self.name = ("like", name)
|
@ -1,3 +1,4 @@
|
||||
from .dict import DictType, DictDetails, DictTypeSimpleOut, DictDetailsSimpleOut, DictTypeSelectOut
|
||||
from .settings_tab import SettingsTab, SettingsTabSimpleOut
|
||||
from .settings import Settings, SettingsSimpleOut
|
||||
from .task import Task, TaskSimpleOut
|
||||
|
33
kinit-api/apps/vadmin/system/schemas/task.py
Normal file
33
kinit-api/apps/vadmin/system/schemas/task.py
Normal file
@ -0,0 +1,33 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/25 15:08
|
||||
# @File : task.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
|
||||
from typing import Optional
|
||||
from pydantic import BaseModel, Field
|
||||
from core.data_types import DatetimeStr, ObjectIdStr
|
||||
|
||||
|
||||
class Task(BaseModel):
|
||||
name: str
|
||||
group: Optional[str] = None
|
||||
job_class: str
|
||||
exec_strategy: str
|
||||
expression: str
|
||||
is_active: Optional[bool] = True # 临时字段,不在表中创建
|
||||
remark: Optional[str] = None
|
||||
start_date: Optional[DatetimeStr] = None
|
||||
end_date: Optional[DatetimeStr] = None
|
||||
|
||||
|
||||
class TaskSimpleOut(Task):
|
||||
id: ObjectIdStr = Field(..., alias='_id')
|
||||
create_datetime: DatetimeStr
|
||||
update_datetime: DatetimeStr
|
||||
last_run_datetime: Optional[DatetimeStr] = None # 临时字段,不在表中创建
|
||||
|
||||
class Config:
|
||||
orm_mode = True
|
@ -9,19 +9,22 @@
|
||||
from typing import List
|
||||
from aioredis import Redis
|
||||
from fastapi import APIRouter, Depends, Body, UploadFile, Form
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from application.settings import ALIYUN_OSS
|
||||
from core.database import db_getter, redis_getter
|
||||
from core.database import db_getter, redis_getter, mongo_getter
|
||||
from utils.file.aliyun_oss import AliyunOSS, BucketConf
|
||||
from utils.file.file_manage import FileManage
|
||||
from utils.response import SuccessResponse, ErrorResponse
|
||||
from utils.sms.code import CodeSMS
|
||||
from utils.tools import generate_string
|
||||
from . import schemas, crud
|
||||
from core.dependencies import IdList
|
||||
from apps.vadmin.auth.utils.current import AllUserAuth, FullAdminAuth, OpenAuth
|
||||
from apps.vadmin.auth.utils.validation.auth import Auth
|
||||
from .params import DictTypeParams, DictDetailParams
|
||||
from .params import DictTypeParams, DictDetailParams, TaskParams
|
||||
from apps.vadmin.auth import crud as vadminAuthCRUD
|
||||
from .params.task import TaskRecordParams
|
||||
|
||||
app = APIRouter()
|
||||
|
||||
@ -150,7 +153,11 @@ async def get_settings_tabs_values(tab_id: int, auth: Auth = Depends(FullAdminAu
|
||||
|
||||
|
||||
@app.put("/settings/tabs/values", summary="更新系统配置信息")
|
||||
async def put_settings_tabs_values(datas: dict = Body(...), auth: Auth = Depends(FullAdminAuth()), rd: Redis = Depends(redis_getter)):
|
||||
async def put_settings_tabs_values(
|
||||
datas: dict = Body(...),
|
||||
auth: Auth = Depends(FullAdminAuth()),
|
||||
rd: Redis = Depends(redis_getter)
|
||||
):
|
||||
return SuccessResponse(await crud.SettingsDal(auth.db).update_datas(datas, rd))
|
||||
|
||||
|
||||
@ -167,3 +174,87 @@ async def get_settings_privacy(auth: Auth = Depends(FullAdminAuth())):
|
||||
@app.get("/settings/agreement", summary="获取用户协议")
|
||||
async def get_settings_agreement(auth: Auth = Depends(FullAdminAuth())):
|
||||
return SuccessResponse((await crud.SettingsDal(auth.db).get_data(config_key="web_agreement")).config_value)
|
||||
|
||||
|
||||
###########################################################
|
||||
# 定时任务管理
|
||||
###########################################################
|
||||
@app.get("/tasks", summary="获取定时任务列表")
|
||||
async def get_tasks(
|
||||
p: TaskParams = Depends(),
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
datas, count = await crud.TaskDal(db).get_tasks(**p.dict())
|
||||
return SuccessResponse(datas, count=count)
|
||||
|
||||
|
||||
@app.post("/tasks", summary="添加定时任务")
|
||||
async def post_tasks(
|
||||
data: schemas.Task,
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
rd: Redis = Depends(redis_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
return SuccessResponse(await crud.TaskDal(db).create_task(rd, data))
|
||||
|
||||
|
||||
@app.put("/tasks", summary="更新定时任务")
|
||||
async def put_tasks(
|
||||
_id: str,
|
||||
data: schemas.Task,
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
rd: Redis = Depends(redis_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
return SuccessResponse(await crud.TaskDal(db).put_task(rd, _id, data))
|
||||
|
||||
|
||||
@app.delete("/tasks", summary="删除单个定时任务")
|
||||
async def delete_task(
|
||||
_id: str,
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
return SuccessResponse(await crud.TaskDal(db).delete_task(_id))
|
||||
|
||||
|
||||
@app.get("/task", summary="获取定时任务详情")
|
||||
async def get_task(
|
||||
_id: str,
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
return SuccessResponse(await crud.TaskDal(db).get_task(_id, v_schema=schemas.TaskSimpleOut))
|
||||
|
||||
|
||||
@app.post("/task", summary="执行一次定时任务")
|
||||
async def run_once_task(
|
||||
_id: str,
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
rd: Redis = Depends(redis_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
return SuccessResponse(await crud.TaskDal(db).run_once_task(rd, _id))
|
||||
|
||||
|
||||
###########################################################
|
||||
# 定时任务分组管理
|
||||
###########################################################
|
||||
@app.get("/task/group/options", summary="获取定时任务分组选择项列表")
|
||||
async def get_task_group_options(db: AsyncIOMotorDatabase = Depends(mongo_getter), auth: Auth = Depends(AllUserAuth())):
|
||||
return SuccessResponse(await crud.TaskGroupDal(db).get_datas(limit=0))
|
||||
|
||||
|
||||
###########################################################
|
||||
# 定时任务调度日志
|
||||
###########################################################
|
||||
@app.get("/task/records", summary="获取定时任务调度日志列表")
|
||||
async def get_task_records(
|
||||
p: TaskRecordParams = Depends(),
|
||||
db: AsyncIOMotorDatabase = Depends(mongo_getter),
|
||||
auth: Auth = Depends(AllUserAuth())
|
||||
):
|
||||
count = await crud.TaskRecordDal(db).get_count(**p.to_count())
|
||||
datas = await crud.TaskRecordDal(db).get_datas(**p.dict())
|
||||
return SuccessResponse(datas, count=count)
|
||||
|
@ -63,7 +63,7 @@ class DalBase:
|
||||
:param v_or: 或逻辑查询
|
||||
:param v_order: 排序,默认正序,为 desc 是倒叙
|
||||
:param v_order_field: 排序字段
|
||||
:param v_return_none: 是否返回空 None,否认 抛出异常,默认抛出异常
|
||||
:param v_return_none: 是否返回空 None,否则抛出异常,默认抛出异常
|
||||
:param v_schema: 指定使用的序列化对象
|
||||
:param kwargs: 查询参数
|
||||
"""
|
||||
|
@ -9,6 +9,9 @@
|
||||
"""
|
||||
自定义数据类型 - 官方文档:https://pydantic-docs.helpmanual.io/usage/types/#custom-data-types
|
||||
"""
|
||||
import datetime
|
||||
|
||||
from bson import ObjectId
|
||||
|
||||
from .validator import *
|
||||
|
||||
@ -23,6 +26,9 @@ class DatetimeStr(str):
|
||||
def validate(cls, v):
|
||||
if isinstance(v, str):
|
||||
return v
|
||||
elif isinstance(v, dict):
|
||||
# 转换为datetime对象
|
||||
v = datetime.datetime.strptime(v.get("$date"), "%Y-%m-%dT%H:%M:%S.%fZ")
|
||||
return v.strftime("%Y-%m-%d %H:%M:%S")
|
||||
|
||||
|
||||
@ -59,3 +65,20 @@ class DateStr(str):
|
||||
if isinstance(v, str):
|
||||
return v
|
||||
return v.strftime("%Y-%m-%d")
|
||||
|
||||
|
||||
class ObjectIdStr(str):
|
||||
|
||||
@classmethod
|
||||
def __get_validators__(cls):
|
||||
yield cls.validate
|
||||
|
||||
@classmethod
|
||||
def validate(cls, v):
|
||||
if isinstance(v, str):
|
||||
return v
|
||||
elif isinstance(v, dict):
|
||||
return v.get("$oid")
|
||||
elif isinstance(v, ObjectId):
|
||||
return str(v)
|
||||
return v
|
||||
|
@ -14,9 +14,10 @@ from aioredis import Redis
|
||||
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession
|
||||
from sqlalchemy.ext.declarative import declared_attr, declarative_base
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from application.settings import SQLALCHEMY_DATABASE_URL, REDIS_DB_ENABLE
|
||||
from application.settings import SQLALCHEMY_DATABASE_URL, REDIS_DB_ENABLE, MONGO_DB_ENABLE
|
||||
from fastapi import Request
|
||||
from core.exception import CustomException
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
|
||||
|
||||
def create_async_engine_session(database_url: str):
|
||||
@ -93,10 +94,22 @@ async def db_getter():
|
||||
|
||||
def redis_getter(request: Request) -> Redis:
|
||||
"""
|
||||
获取关系数据库
|
||||
获取 redis 数据库对象
|
||||
|
||||
数据库依赖项,它将在单个请求中使用,然后在请求完成后将其关闭。
|
||||
全局挂载,使用一个数据库对象
|
||||
"""
|
||||
if not REDIS_DB_ENABLE:
|
||||
raise CustomException("请先配置Redis数据库链接并启用!", desc="请启用 application/settings.py: REDIS_DB_ENABLE")
|
||||
return request.app.state.redis
|
||||
|
||||
|
||||
def mongo_getter(request: Request) -> AsyncIOMotorDatabase:
|
||||
"""
|
||||
获取 mongo 数据库对象
|
||||
|
||||
全局挂载,使用一个数据库对象
|
||||
"""
|
||||
if not MONGO_DB_ENABLE:
|
||||
raise CustomException(msg="请先开启 MongoDB 数据库连接!", desc="请启用 application/settings.py: MONGO_DB_ENABLE")
|
||||
return request.app.state.mongo
|
||||
|
||||
|
@ -8,8 +8,8 @@
|
||||
|
||||
|
||||
from fastapi import FastAPI
|
||||
from motor.motor_asyncio import AsyncIOMotorClient
|
||||
from application.settings import REDIS_DB_URL, MONGO_DB_URL, MONGO_DB_NAME, EVENTS
|
||||
from core.mongo import db
|
||||
from utils.cache import Cache
|
||||
import aioredis
|
||||
from contextlib import asynccontextmanager
|
||||
@ -82,11 +82,13 @@ async def connect_mongo(app: FastAPI, status: bool):
|
||||
:return:
|
||||
"""
|
||||
if status:
|
||||
client: AsyncIOMotorClient = AsyncIOMotorClient(MONGO_DB_URL, maxPoolSize=10, minPoolSize=10)
|
||||
app.state.mongo_client = client
|
||||
app.state.mongo = client[MONGO_DB_NAME]
|
||||
print("Connecting to Mongo")
|
||||
await db.connect_to_database(path=MONGO_DB_URL, db_name=MONGO_DB_NAME)
|
||||
else:
|
||||
print("Mongo connection closed")
|
||||
await db.close_database_connection()
|
||||
app.state.mongo_client.close()
|
||||
|
||||
|
||||
|
||||
|
@ -19,8 +19,9 @@ from fastapi.routing import APIRoute
|
||||
from user_agents import parse
|
||||
from application.settings import OPERATION_RECORD_METHOD, MONGO_DB_ENABLE, IGNORE_OPERATION_FUNCTION,\
|
||||
DEMO_WHITE_LIST_PATH, DEMO
|
||||
from core.mongo import get_database
|
||||
from utils.response import ErrorResponse
|
||||
from apps.vadmin.record.crud import OperationRecordDal
|
||||
from core.database import mongo_getter
|
||||
|
||||
|
||||
def write_request_log(request: Request, response: Response):
|
||||
@ -112,8 +113,7 @@ def register_operation_record_middleware(app: FastAPI):
|
||||
"create_datetime": datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"params": json.dumps(params)
|
||||
}
|
||||
db = await get_database()
|
||||
await db.create_data("operation_record", document)
|
||||
await OperationRecordDal(mongo_getter(request)).create_data(document)
|
||||
return response
|
||||
|
||||
|
||||
|
@ -1,13 +0,0 @@
|
||||
from .database_manage import DatabaseManage
|
||||
from .mongo_manage import MongoManage
|
||||
from application.settings import MONGO_DB_ENABLE
|
||||
from core.exception import CustomException
|
||||
from utils import status
|
||||
|
||||
db = MongoManage()
|
||||
|
||||
|
||||
async def get_database() -> DatabaseManage:
|
||||
if not MONGO_DB_ENABLE:
|
||||
raise CustomException(msg="请先开启 MongoDB 数据库连接!", code=status.HTTP_ERROR)
|
||||
return db
|
@ -1,47 +0,0 @@
|
||||
from abc import abstractmethod
|
||||
from typing import Any
|
||||
|
||||
|
||||
class DatabaseManage:
|
||||
"""
|
||||
This class is meant to be extended from
|
||||
./mongo_manage.py which will be the actual connection to mongodb.
|
||||
"""
|
||||
|
||||
@property
|
||||
def client(self):
|
||||
raise NotImplementedError
|
||||
|
||||
@property
|
||||
def db(self):
|
||||
raise NotImplementedError
|
||||
|
||||
# database connect and close connections
|
||||
@abstractmethod
|
||||
async def connect_to_database(self, path: str, db_name: str):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def close_database_connection(self):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def create_data(self, collection: str, data: dict):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def get_datas(
|
||||
self,
|
||||
collection: str,
|
||||
page: int = 1,
|
||||
limit: int = 10,
|
||||
v_schema: Any = None,
|
||||
v_order: str = None,
|
||||
v_order_field: str = None,
|
||||
**kwargs
|
||||
):
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def get_count(self, collection: str, **kwargs) -> int:
|
||||
pass
|
@ -1,82 +0,0 @@
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from bson.json_util import dumps
|
||||
from motor.motor_asyncio import AsyncIOMotorClient, AsyncIOMotorDatabase
|
||||
from core.mongo import DatabaseManage
|
||||
from pymongo.results import InsertOneResult
|
||||
|
||||
|
||||
class MongoManage(DatabaseManage):
|
||||
"""
|
||||
This class extends from ./database_manage.py
|
||||
which have the abstract methods to be re-used here.
|
||||
博客:https://www.cnblogs.com/aduner/p/13532504.html
|
||||
mongodb 官网:https://www.mongodb.com/docs/drivers/motor/
|
||||
motor 文档:https://motor.readthedocs.io/en/stable/
|
||||
"""
|
||||
|
||||
client: AsyncIOMotorClient = None
|
||||
db: AsyncIOMotorDatabase = None
|
||||
|
||||
async def connect_to_database(self, path: str, db_name: str):
|
||||
self.client = AsyncIOMotorClient(path, maxPoolSize=10, minPoolSize=10)
|
||||
self.db = self.client[db_name]
|
||||
|
||||
async def close_database_connection(self):
|
||||
self.client.close()
|
||||
|
||||
async def create_data(self, collection: str, data: dict) -> InsertOneResult:
|
||||
return await self.db[collection].insert_one(data)
|
||||
|
||||
async def get_datas(
|
||||
self,
|
||||
collection: str,
|
||||
page: int = 1,
|
||||
limit: int = 10,
|
||||
v_schema: Any = None,
|
||||
v_order: str = None,
|
||||
v_order_field: str = None,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
使用 find() 要查询的一组文档。 find() 没有I / O,也不需要 await 表达式。它只是创建一个 AsyncIOMotorCursor 实例
|
||||
当您调用 to_list() 或为循环执行异步时 (async for) ,查询实际上是在服务器上执行的。
|
||||
"""
|
||||
|
||||
params = self.filter_condition(**kwargs)
|
||||
cursor = self.db[collection].find(params)
|
||||
|
||||
# 对查询应用排序(sort),跳过(skip)或限制(limit)
|
||||
cursor.sort("create_datetime", -1).skip((page - 1) * limit).limit(limit)
|
||||
|
||||
datas = []
|
||||
async for row in cursor:
|
||||
del row['_id']
|
||||
data = json.loads(dumps(row))
|
||||
if v_schema:
|
||||
data = v_schema.parse_obj(data).dict()
|
||||
datas.append(data)
|
||||
return datas
|
||||
|
||||
async def get_count(self, collection: str, **kwargs) -> int:
|
||||
params = self.filter_condition(**kwargs)
|
||||
return await self.db[collection].count_documents(params)
|
||||
|
||||
@classmethod
|
||||
def filter_condition(cls, **kwargs):
|
||||
"""
|
||||
过滤条件
|
||||
"""
|
||||
params = {}
|
||||
for k, v in kwargs.items():
|
||||
if not v:
|
||||
continue
|
||||
elif isinstance(v, tuple):
|
||||
if v[0] == "like" and v[1]:
|
||||
params[k] = {'$regex': v[1]}
|
||||
elif v[0] == "between" and len(v[1]) == 2:
|
||||
params[k] = {'$gte': f"{v[1][0]} 00:00:00", '$lt': f"{v[1][1]} 23:59:59"}
|
||||
else:
|
||||
params[k] = v
|
||||
return params
|
165
kinit-api/core/mongo_manage.py
Normal file
165
kinit-api/core/mongo_manage.py
Normal file
@ -0,0 +1,165 @@
|
||||
import datetime
|
||||
import json
|
||||
from typing import Any
|
||||
from bson import ObjectId
|
||||
from bson.errors import InvalidId
|
||||
from bson.json_util import dumps
|
||||
from fastapi.encoders import jsonable_encoder
|
||||
from motor.motor_asyncio import AsyncIOMotorDatabase
|
||||
from pymongo.results import InsertOneResult, UpdateResult
|
||||
from core.exception import CustomException
|
||||
from utils import status
|
||||
|
||||
|
||||
class MongoManage:
|
||||
"""
|
||||
mongodb 数据库管理器
|
||||
博客:https://www.cnblogs.com/aduner/p/13532504.html
|
||||
mongodb 官网:https://www.mongodb.com/docs/drivers/motor/
|
||||
motor 文档:https://motor.readthedocs.io/en/stable/
|
||||
"""
|
||||
|
||||
# 倒叙
|
||||
ORDER_FIELD = ["desc", "descending"]
|
||||
|
||||
def __init__(self, db: AsyncIOMotorDatabase, collection: str, schema: Any = None, is_object_id: bool = True):
|
||||
self.db = db
|
||||
self.collection = db[collection]
|
||||
self.schema = schema
|
||||
self.is_object_id = is_object_id
|
||||
|
||||
async def get_data(
|
||||
self,
|
||||
_id: str = None,
|
||||
v_return_none: bool = False,
|
||||
v_schema: Any = None,
|
||||
**kwargs
|
||||
) -> dict | None:
|
||||
"""
|
||||
获取单个数据,默认使用 ID 查询,否则使用关键词查询
|
||||
|
||||
:param _id: 数据 ID
|
||||
:param v_return_none: 是否返回空 None,否则抛出异常,默认抛出异常
|
||||
:param v_schema: 指定使用的序列化对象
|
||||
"""
|
||||
if _id and self.is_object_id:
|
||||
kwargs["_id"] = ObjectId(_id)
|
||||
params = self.filter_condition(**kwargs)
|
||||
data = await self.collection.find_one(params)
|
||||
if not data and v_return_none:
|
||||
return None
|
||||
elif not data:
|
||||
raise CustomException("查找失败,未查找到对应数据", code=status.HTTP_404_NOT_FOUND)
|
||||
elif data and v_schema:
|
||||
return jsonable_encoder(v_schema(**data))
|
||||
return data
|
||||
|
||||
async def create_data(self, data: dict | Any) -> InsertOneResult:
|
||||
"""
|
||||
创建数据
|
||||
"""
|
||||
if not isinstance(data, dict):
|
||||
data = jsonable_encoder(data)
|
||||
data['create_datetime'] = datetime.datetime.now()
|
||||
data['update_datetime'] = datetime.datetime.now()
|
||||
result = await self.collection.insert_one(data)
|
||||
# 判断插入是否成功
|
||||
if result.acknowledged:
|
||||
return result
|
||||
else:
|
||||
raise CustomException("创建新数据失败", code=status.HTTP_ERROR)
|
||||
|
||||
async def put_data(self, _id: str, data: dict | Any) -> UpdateResult:
|
||||
"""
|
||||
更新数据
|
||||
"""
|
||||
if not isinstance(data, dict):
|
||||
data = jsonable_encoder(data)
|
||||
new_data = {'$set': data}
|
||||
result = await self.collection.update_one({'_id': ObjectId(_id) if self.is_object_id else _id}, new_data)
|
||||
|
||||
if result.matched_count > 0:
|
||||
return result
|
||||
else:
|
||||
raise CustomException("更新失败,未查找到对应数据", code=status.HTTP_404_NOT_FOUND)
|
||||
|
||||
async def delete_data(self, _id: str):
|
||||
"""
|
||||
删除数据
|
||||
"""
|
||||
result = await self.collection.delete_one({'_id': ObjectId(_id) if self.is_object_id else _id})
|
||||
|
||||
if result.deleted_count > 0:
|
||||
return True
|
||||
else:
|
||||
raise CustomException("删除失败,未查找到对应数据", code=status.HTTP_404_NOT_FOUND)
|
||||
|
||||
async def get_datas(
|
||||
self,
|
||||
page: int = 1,
|
||||
limit: int = 10,
|
||||
v_schema: Any = None,
|
||||
v_order: str = None,
|
||||
v_order_field: str = None,
|
||||
v_return_objs: bool = False,
|
||||
**kwargs
|
||||
):
|
||||
"""
|
||||
使用 find() 要查询的一组文档。 find() 没有I / O,也不需要 await 表达式。它只是创建一个 AsyncIOMotorCursor 实例
|
||||
当您调用 to_list() 或为循环执行异步时 (async for) ,查询实际上是在服务器上执行的。
|
||||
"""
|
||||
|
||||
params = self.filter_condition(**kwargs)
|
||||
cursor = self.collection.find(params)
|
||||
|
||||
if v_order or v_order_field:
|
||||
v_order_field = v_order_field if v_order_field else 'create_datetime'
|
||||
v_order = -1 if v_order in self.ORDER_FIELD else 1
|
||||
cursor.sort(v_order_field, v_order)
|
||||
|
||||
if limit != 0:
|
||||
# 对查询应用排序(sort),跳过(skip)或限制(limit)
|
||||
cursor.skip((page - 1) * limit).limit(limit)
|
||||
|
||||
datas = []
|
||||
async for row in cursor:
|
||||
data = json.loads(dumps(row))
|
||||
datas.append(data)
|
||||
|
||||
if not datas or v_return_objs:
|
||||
return datas
|
||||
elif v_schema:
|
||||
datas = [jsonable_encoder(v_schema(**data)) for data in datas]
|
||||
elif self.schema:
|
||||
datas = [jsonable_encoder(self.schema(**data)) for data in datas]
|
||||
return datas
|
||||
|
||||
async def get_count(self, **kwargs) -> int:
|
||||
"""
|
||||
获取统计数据
|
||||
"""
|
||||
params = self.filter_condition(**kwargs)
|
||||
return await self.collection.count_documents(params)
|
||||
|
||||
@classmethod
|
||||
def filter_condition(cls, **kwargs):
|
||||
"""
|
||||
过滤条件
|
||||
"""
|
||||
params = {}
|
||||
for k, v in kwargs.items():
|
||||
if not v:
|
||||
continue
|
||||
elif isinstance(v, tuple):
|
||||
if v[0] == "like" and v[1]:
|
||||
params[k] = {'$regex': v[1]}
|
||||
elif v[0] == "between" and len(v[1]) == 2:
|
||||
params[k] = {'$gte': f"{v[1][0]} 00:00:00", '$lt': f"{v[1][1]} 23:59:59"}
|
||||
elif v[0] == "ObjectId" and v[1]:
|
||||
try:
|
||||
params[k] = ObjectId(v[1])
|
||||
except InvalidId:
|
||||
raise CustomException("任务编号格式不正确!")
|
||||
else:
|
||||
params[k] = v
|
||||
return params
|
@ -104,4 +104,4 @@ if __name__ == '__main__':
|
||||
# print(generate_invitation_code())
|
||||
# print(int(datetime.datetime.now().timestamp()))
|
||||
# print(datetime.datetime.today() + datetime.timedelta(days=7))
|
||||
print(generate_string())
|
||||
print(generate_string(15))
|
||||
|
32
kinit-task/.gitignore
vendored
Normal file
32
kinit-task/.gitignore
vendored
Normal file
@ -0,0 +1,32 @@
|
||||
# Editor directories and files
|
||||
.idea/
|
||||
.idea
|
||||
.vscode
|
||||
*.suo
|
||||
*.ntvs*
|
||||
*.njsproj
|
||||
*.sln
|
||||
logs/*
|
||||
!logs/.gitkeep
|
||||
temp/*
|
||||
!temp/.gitkeep
|
||||
!static/.gitkeep
|
||||
!alembic/versions/.gitkeep
|
||||
|
||||
# dotenv
|
||||
.env
|
||||
|
||||
# virtualenv
|
||||
venv/
|
||||
ENV/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
*.db
|
||||
.DS_Store
|
||||
__pycache__
|
||||
!migrations/__init__.py
|
||||
*.pyc
|
410
kinit-task/README.md
Normal file
410
kinit-task/README.md
Normal file
@ -0,0 +1,410 @@
|
||||
|
||||
|
||||
# kinit-task
|
||||
|
||||
**感谢若依:http://demo.ruoyi.vip/**
|
||||
|
||||
|
||||
|
||||
定时任务功能:
|
||||
|
||||
- [x] 支持添加四种定时任务:
|
||||
- [x] 添加 date 指定日期时间执行定时任务
|
||||
- [x] 添加 Cron 表达式定时任务
|
||||
- [x] 添加 Interval 时间间隔定时任务
|
||||
- [x] 支持立即执行任务功能
|
||||
- [x] 使用 redis 消息队列功能动态添加任务
|
||||
- [x] 使用 mongodb 数据库存储持久化保存任务
|
||||
- [x] 任务表达式使用类路径表示,支持添加初始化参数:支持字符串,布尔类型,长整型,浮点型,整型
|
||||
|
||||
- [x] 每次任务执行完成后,记录日志到 mongodb 数据中:开始/结束执行时间,耗时,任务返回值,异常信息
|
||||
|
||||
|
||||
|
||||
## 使用
|
||||
|
||||
1. 安装依赖
|
||||
|
||||
```
|
||||
# 安装依赖库
|
||||
pip3 install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# 第三方源:
|
||||
1. 阿里源: https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
2. 修改项目数据库配置信息
|
||||
|
||||
在 `application/config` 目录中
|
||||
|
||||
- development.py:开发环境
|
||||
|
||||
- production.py:生产环境
|
||||
|
||||
```python
|
||||
"""
|
||||
MongoDB 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
MONGO_DB_NAME = "数据库名称"
|
||||
MONGO_DB_URL = f"mongodb://用户名:密码@地址:端口/?authSource={MONGO_DB_NAME}"
|
||||
|
||||
|
||||
"""
|
||||
Redis 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
REDIS_DB_URL = "redis://:密码@地址:端口/数据库名称"
|
||||
```
|
||||
|
||||
3. 启动
|
||||
|
||||
```
|
||||
python3 main.py
|
||||
```
|
||||
|
||||
|
||||
|
||||
## APScheduler
|
||||
|
||||
官方文档:https://apscheduler.readthedocs.io/en/master/userguide.html
|
||||
|
||||
Github:https://github.com/agronholm/apscheduler
|
||||
|
||||
PYPI:https://pypi.org/project/APScheduler/
|
||||
|
||||
|
||||
|
||||
安装/更新
|
||||
|
||||
```
|
||||
pip install -U APScheduler -i https://mirrors.aliyun.com/pypi/simple/
|
||||
```
|
||||
|
||||
|
||||
|
||||
### 使用
|
||||
|
||||
```
|
||||
# 添加任务
|
||||
|
||||
from apscheduler.schedulers.blocking import BlockingScheduler
|
||||
|
||||
def job():
|
||||
print('Hello world!')
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler.add_job(job, 'interval', minutes=1)
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
```
|
||||
# 立即执行
|
||||
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
def job():
|
||||
print("Hello, world!")
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
|
||||
# 立即执行任务
|
||||
scheduler.add_job(job, next_run_time=datetime.now(), id='my_job')
|
||||
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
```
|
||||
# 判断是否存在
|
||||
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
def my_job():
|
||||
print('Hello, world!')
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
scheduler.add_job(my_job, 'interval', seconds=10, id='my_job')
|
||||
scheduler.start()
|
||||
|
||||
# 检查任务是否存在
|
||||
if scheduler.get_job('my_job'):
|
||||
print('任务存在')
|
||||
else:
|
||||
print('任务不存在')
|
||||
```
|
||||
|
||||
```
|
||||
# 删除任务
|
||||
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
def my_job():
|
||||
print('Hello, world!')
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
scheduler.add_job(my_job, 'interval', seconds=10, id='my_job')
|
||||
scheduler.start()
|
||||
|
||||
# 删除任务
|
||||
scheduler.remove_job('my_job')
|
||||
```
|
||||
|
||||
```
|
||||
# 添加参数
|
||||
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
def job(arg1, arg2):
|
||||
print('This is a job with arguments: {}, {}'.format(arg1, arg2))
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
scheduler.add_job(job, 'interval', seconds=5, args=('hello', 'world'))
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
```
|
||||
# 获取当前正在执行的任务列表
|
||||
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
def job():
|
||||
print('This is a job.')
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
scheduler.add_job(job, 'interval', seconds=5)
|
||||
scheduler.start()
|
||||
|
||||
# 获取当前正在执行的任务列表
|
||||
jobs = scheduler.get_jobs()
|
||||
for job in jobs:
|
||||
print(job)
|
||||
```
|
||||
|
||||
### 添加定时任务 add_job 方法
|
||||
|
||||
APScheduler的`add_job`方法用于添加定时任务。除了使用Cron表达式来指定定时任务的调度规则之外,`add_job`方法还支持其他几种方法来设置定时任务的执行时间。以下是`add_job`方法常用的几种调度方式:
|
||||
|
||||
- date:指定一个具体的日期和时间来执行任务。
|
||||
|
||||
```python
|
||||
scheduler.add_job(job_function, 'date', run_date='2023-06-30 12:00:00')
|
||||
```
|
||||
|
||||
在上述示例中,任务将在指定的日期和时间(2023年6月30日12:00:00)执行。
|
||||
|
||||
- interval:指定一个时间间隔来执行任务。
|
||||
|
||||
```python
|
||||
scheduler.add_job(job_function, 'interval', minutes=30)
|
||||
```
|
||||
|
||||
在上述示例中,任务将每隔30分钟执行一次。
|
||||
|
||||
- cron:使用Cron表达式来指定任务的执行时间。
|
||||
|
||||
```python
|
||||
scheduler.add_job(job_function, 'cron', hour=8, minute=0, day_of_week='0-4')
|
||||
```
|
||||
|
||||
在上述示例中,任务将在每个工作日的早上8点执行。
|
||||
|
||||
- timedelta:指定一个时间间隔来执行任务,但相对于当前时间的偏移量。
|
||||
|
||||
```python
|
||||
from datetime import timedelta
|
||||
|
||||
scheduler.add_job(job_function, 'interval', seconds=10, start_date=datetime.now() + timedelta(seconds=5))
|
||||
```
|
||||
|
||||
在上述示例中,任务将在当前时间的5秒后开始执行,然后每隔10秒执行一次。
|
||||
|
||||
这些方法提供了不同的方式来安排定时任务的执行时间。你可以根据具体需求选择适合的调度方式,并结合相关参数来设置定时任务的执行规则。无论使用哪种方法,都可以通过`add_job`方法将任务添加到调度器中,以便按照预定的时间规则执行任务。
|
||||
|
||||
### cron 触发器
|
||||
|
||||
`cron`触发器是`APScheduler`中常用的一种触发器类型,用于基于cron表达式来触发任务。它提供了灵活且精确的任务调度规则,可以在特定的日期和时间点上触发任务。
|
||||
|
||||
以下是关于`cron`触发器的详细解释:
|
||||
|
||||
1. **创建触发器:**要创建一个`cron`触发器,可以使用`CronTrigger`类并指定cron表达式作为参数。cron表达式是一种字符串格式,用于指定任务触发的时间规则。它由多个字段组成,每个字段表示时间的不同部分,例如分钟、小时、日期等。示例代码如下:
|
||||
|
||||
```python
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
|
||||
# 创建每天上午10点触发的cron触发器
|
||||
trigger = CronTrigger(hour=10)
|
||||
```
|
||||
|
||||
在上述示例中,我们创建了一个每天上午10点触发的`cron`触发器。
|
||||
|
||||
2. **添加触发器到任务:**创建触发器后,可以将它与任务相关联,以定义任务的调度规则。可以使用`add_job()`方法的`trigger`参数将触发器添加到任务中。示例代码如下:
|
||||
|
||||
```python
|
||||
from apscheduler.schedulers.blocking import BlockingScheduler
|
||||
|
||||
def job_function():
|
||||
# 任务逻辑
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler.add_job(job_function, trigger=CronTrigger(hour=10))
|
||||
```
|
||||
|
||||
在上述示例中,我们将`cron`触发器添加到了名为`job_function`的任务中,使得该任务在每天上午10点触发。
|
||||
|
||||
3. **cron表达式:**cron表达式由多个字段组成,用空格分隔。每个字段表示时间的不同部分,具体如下:
|
||||
|
||||
- `分钟`:范围是0-59。
|
||||
- `小时`:范围是0-23。
|
||||
- `日期`:范围是1-31。
|
||||
- `月份`:范围是1-12。
|
||||
- `星期几`:范围是0-6,其中0表示星期日,1表示星期一,以此类推。
|
||||
|
||||
通过在cron表达式中指定相应的字段值,可以创建各种复杂的调度规则。例如:`0 12 * * *`表示每天中午12点触发,`0 8-18 * * MON-FRI`表示工作日每小时从早上8点到下午6点之间的整点触发。
|
||||
|
||||
```python
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
|
||||
# 创建每周一至周五上午10点触发的cron触发器
|
||||
trigger = CronTrigger(hour=10, day_of_week='mon-fri')
|
||||
```
|
||||
|
||||
在上述示例中,我们创建了一个每周一至周五上午10点触发的`cron`触发
|
||||
|
||||
### interval 触发器
|
||||
|
||||
`interval`触发器是`APScheduler`中常用的一种触发器类型,用于在固定的时间间隔内重复触发任务。它基于时间间隔而不是具体的日期和时间来触发任务,适用于需要以固定间隔执行的周期性任务。
|
||||
|
||||
以下是关于`interval`触发器的详细解释:
|
||||
|
||||
1. **创建触发器:**要创建一个`interval`触发器,可以使用`IntervalTrigger`类并指定时间间隔参数。时间间隔可以以秒、分钟、小时或者天为单位进行设置。示例代码如下:
|
||||
|
||||
```python
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
|
||||
# 创建每5秒触发一次的interval触发器
|
||||
trigger = IntervalTrigger(seconds=5)
|
||||
```
|
||||
|
||||
在上述示例中,我们创建了一个每5秒触发一次的`interval`触发器。
|
||||
|
||||
2. **添加触发器到任务:**创建触发器后,可以将它与任务相关联,以定义任务的调度规则。可以使用`add_job()`方法的`trigger`参数将触发器添加到任务中。示例代码如下:
|
||||
|
||||
```python
|
||||
from apscheduler.schedulers.blocking import BlockingScheduler
|
||||
|
||||
def job_function():
|
||||
# 任务逻辑
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler.add_job(job_function, trigger=IntervalTrigger(seconds=5))
|
||||
```
|
||||
|
||||
在上述示例中,我们将`interval`触发器添加到了名为`job_function`的任务中,使得该任务每隔5秒触发一次。
|
||||
|
||||
3. **触发器选项:**`IntervalTrigger`类还提供了其他可选的参数,用于进一步定制触发器的行为,例如:
|
||||
|
||||
- `start_date`:指定触发器的开始日期和时间。
|
||||
- `end_date`:指定触发器的结束日期和时间。
|
||||
- `timezone`:指定触发器的时区。
|
||||
|
||||
这些选项可以通过在创建触发器时传递相应的参数来设置。
|
||||
|
||||
```python
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
from datetime import datetime, timedelta
|
||||
from pytz import timezone
|
||||
|
||||
# 创建每5分钟触发一次的interval触发器,从指定日期开始,结束日期为一周后
|
||||
start_date = datetime(2023, 1, 1, 0, 0, 0)
|
||||
end_date = start_date + timedelta(weeks=1)
|
||||
tz = timezone('US/Eastern')
|
||||
trigger = IntervalTrigger(minutes=5, start_date=start_date, end_date=end_date, timezone=tz)
|
||||
```
|
||||
|
||||
在上述示例中,我们创建了一个每5分钟触发一次的`interval`触发器,并指定了开始日期、结束日期和时区。
|
||||
|
||||
通过使用`interval`触发器,可以方便地设置任务以固定的时间间隔重复执行。可以根据实际需求设置触发器的参数,以满足不同的周期性任务调度要求。
|
||||
|
||||
### 事件监听器
|
||||
|
||||
事件监听器是`APScheduler`中的一种机制,用于监视和响应调度器中的事件。当特定事件发生时,监听器将执行预定义的操作,例如记录日志、发送通知或执行自定义逻辑。下面是关于`APScheduler`事件监听器的详细解释:
|
||||
|
||||
1. **事件类型:**`APScheduler`中有多个事件类型,每个事件类型对应着不同的调度器行为或状态变化。一些常见的事件类型包括:
|
||||
- `EVENT_JOB_ADDED`:当添加新任务时触发。
|
||||
- `EVENT_JOB_REMOVED`:当移除任务时触发。
|
||||
- `EVENT_JOB_MODIFIED`:当修改任务时触发。
|
||||
- `EVENT_JOB_EXECUTED`:当任务执行完成时触发。
|
||||
- `EVENT_JOB_ERROR`:当任务执行出错时触发。
|
||||
- `EVENT_SCHEDULER_STARTED`:当调度器启动时触发。
|
||||
- `EVENT_SCHEDULER_SHUTDOWN`:当调度器关闭时触发。
|
||||
|
||||
### max_instances 最大实例
|
||||
|
||||
报错:
|
||||
|
||||
```
|
||||
Execution of job "Test.main (trigger: interval[0:00:01], next run at: 2023-06-21 16:23:23 CST)" skipped: maximum number of running instances reached (1)
|
||||
```
|
||||
|
||||
这个错误提示表示当前正在运行的任务实例数量已达到最大限制,因此调度器跳过了该任务的执行。该错误通常是由于设置了并发执行任务的限制导致的。
|
||||
|
||||
默认情况下,`APScheduler`允许同时运行的任务实例数量是1,也就是说同一任务在执行时不会与自身的其他实例并发执行。这是为了防止任务并发执行引发的问题,例如资源冲突或意外的行为。
|
||||
|
||||
如果你想要允许任务并发执行,可以通过调整调度器的配置来增加最大并发实例数量。在创建调度器时,可以传递`max_instances`参数来设置最大并发实例数量。示例如下:
|
||||
|
||||
```python
|
||||
from apscheduler.schedulers.blocking import BlockingScheduler
|
||||
|
||||
def job_function():
|
||||
# 任务逻辑
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler.add_job(job_function, 'interval', seconds=1, max_instances=5) # 设置最大并发实例为5
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
在上述示例中,`max_instances`参数被设置为5,表示同一任务可以最多同时运行5个实例。如果同时触发了多个任务实例,调度器将会并发执行它们,直到达到最大实例数量为止。
|
||||
|
||||
请注意,增加最大并发实例数量可能会增加系统负载和资源消耗。因此,在设置并发执行之前,请确保你的系统能够处理额外的负载,并根据实际需求进行适当的配置。
|
||||
|
||||
### 如何达到了最大实例,那么超过最大实例的其他任务会延后执行吗
|
||||
|
||||
当调度器的任务实例数量达到最大限制时,超过最大实例数量的其他任务将会被延后执行。`APScheduler`会自动管理任务的执行队列,并按照任务的调度规则和最大实例限制进行调度。
|
||||
|
||||
具体来说,当有新的任务触发时,调度器会检查当前正在运行的任务实例数量是否已经达到最大限制。如果是,则该任务的执行会被延后,直到有可用的执行实例。延后执行的任务会继续保持在调度器的执行队列中,等待下一个可用的执行时机。
|
||||
|
||||
值得注意的是,任务的延后执行是相对于其原定的调度时间而言的。也就是说,即使一个任务被延后执行,它仍然会尽可能地在其下一个调度时间点之前执行,以保持任务的调度准确性。
|
||||
|
||||
以下是一个示例,演示了当任务实例数量达到最大限制时,其他任务会被延后执行:
|
||||
|
||||
```python
|
||||
from apscheduler.schedulers.blocking import BlockingScheduler
|
||||
|
||||
def job_function():
|
||||
# 任务逻辑
|
||||
|
||||
scheduler = BlockingScheduler()
|
||||
scheduler.add_job(job_function, 'interval', seconds=10, max_instances=2) # 设置最大并发实例为2
|
||||
|
||||
# 添加多个任务,超过最大实例数量
|
||||
for i in range(5):
|
||||
scheduler.add_job(job_function, 'interval', seconds=5)
|
||||
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
在上述示例中,我们设置了最大并发实例数量为2,然后添加了5个间隔为5秒的任务。由于最大实例数量限制为2,所以前两个任务会立即开始执行,而后面的三个任务会被延后执行,等待前面的任务完成并释放实例后才能执行。
|
||||
|
||||
总结起来,超过最大实例数量的任务会被放入调度器的执行队列中,并在合适的时机进行延后执行,以保证任务的调度准确性和最大实例限制的有效性。
|
||||
|
||||
## Redis
|
||||
|
||||
### 什么是Redis消息订阅与发布
|
||||
|
||||
Redis是一个开源的高性能内存数据库,支持数据结构丰富、灵活的持久化、复制以及分片。而Redis消息订阅与发布就是Redis中的一种发布/订阅模型,调用方可以订阅某个频道并接收消息,也可以向某个频道发送消息,同时所有订阅该频道的调用方都可以收到这个消息。该模型使用了发布/订阅的方式来实现多个调用方之间的信息通信。
|
||||
|
||||
### Redis消息保留策略
|
||||
|
||||
Redis提供了两种消息保留策略,分别是无保留和有保留。无保留是指当发送者向某个频道发送消息时,如果没有订阅该频道的调用方,就直接将该消息丢弃。而有保留则是指Redis能够将消息保存下来,直到有订阅该频道的调用方出现时再将该消息发送给该调用方。在Redis中,消息保留策略由两个参数来控制,即PUBLISH命令的NX、EX和PX选项。NX选项表示只有当至少有一个订阅者收到消息时,该消息才会被保留。EX和PX选项用于控制保留策略的时间,EX表示以秒为单位的保留时间,PX表示以毫秒为单位的保留时间。开发者可以根据实际需要配置相应的保留策略。
|
7
kinit-task/application/__init__.py
Normal file
7
kinit-task/application/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 13:39
|
||||
# @File : __init__.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
0
kinit-task/application/config/__init__.py
Normal file
0
kinit-task/application/config/__init__.py
Normal file
23
kinit-task/application/config/development.py
Normal file
23
kinit-task/application/config/development.py
Normal file
@ -0,0 +1,23 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2021/10/19 15:47
|
||||
# @File : development.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 数据库生产配置文件
|
||||
|
||||
|
||||
"""
|
||||
MongoDB 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
MONGO_DB_NAME = "数据库名称"
|
||||
MONGO_DB_URL = f"mongodb://用户名:密码@地址:端口/?authSource={MONGO_DB_NAME}"
|
||||
|
||||
|
||||
"""
|
||||
Redis 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
REDIS_DB_URL = "redis://:密码@地址:端口/数据库名称"
|
23
kinit-task/application/config/production.py
Normal file
23
kinit-task/application/config/production.py
Normal file
@ -0,0 +1,23 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2021/10/19 15:47
|
||||
# @File : production.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 数据库开发配置文件
|
||||
|
||||
|
||||
"""
|
||||
MongoDB 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
MONGO_DB_NAME = "数据库名称"
|
||||
MONGO_DB_URL = f"mongodb://用户名:密码@地址:端口/?authSource={MONGO_DB_NAME}"
|
||||
|
||||
|
||||
"""
|
||||
Redis 数据库配置
|
||||
|
||||
与接口是同一个数据库
|
||||
"""
|
||||
REDIS_DB_URL = "redis://:密码@地址:端口/数据库名称"
|
51
kinit-task/application/settings.py
Normal file
51
kinit-task/application/settings.py
Normal file
@ -0,0 +1,51 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 13:39
|
||||
# @File : settings.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
|
||||
import os
|
||||
|
||||
"""项目根目录"""
|
||||
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
|
||||
DEBUG = True
|
||||
|
||||
|
||||
"""
|
||||
引入数据库配置
|
||||
"""
|
||||
if DEBUG:
|
||||
from application.config.development import *
|
||||
else:
|
||||
from application.config.production import *
|
||||
|
||||
|
||||
"""
|
||||
发布/订阅通道
|
||||
|
||||
与接口相互关联,请勿随意更改
|
||||
"""
|
||||
SUBSCRIBE = 'kinit_queue'
|
||||
|
||||
|
||||
"""
|
||||
MongoDB 集合
|
||||
|
||||
与接口相互关联,相互查询,请勿随意更改
|
||||
"""
|
||||
# 用于存放任务调用日志
|
||||
SCHEDULER_TASK_RECORD = "scheduler_task_record"
|
||||
# 用于存放运行中的任务
|
||||
SCHEDULER_TASK_JOBS = "scheduler_task_jobs"
|
||||
# 用于存放任务信息
|
||||
SCHEDULER_TASK = "vadmin_system_task"
|
||||
|
||||
|
||||
"""
|
||||
定时任务脚本目录
|
||||
"""
|
||||
TASKS_ROOT = "tasks"
|
7
kinit-task/core/__init__.py
Normal file
7
kinit-task/core/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:10
|
||||
# @File : __init__.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
57
kinit-task/core/listener.py
Normal file
57
kinit-task/core/listener.py
Normal file
@ -0,0 +1,57 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 14:42
|
||||
# @File : listener.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
|
||||
import datetime
|
||||
import json
|
||||
from apscheduler.events import JobExecutionEvent
|
||||
from core.mongo import get_database
|
||||
import pytz
|
||||
from application.settings import SCHEDULER_TASK_RECORD, SCHEDULER_TASK
|
||||
from core.logger import logger
|
||||
|
||||
|
||||
def before_job_execution(event: JobExecutionEvent):
|
||||
# print("在执行定时任务前执行的代码...")
|
||||
shanghai_tz = pytz.timezone("Asia/Shanghai")
|
||||
start_time: datetime.datetime = event.scheduled_run_time.astimezone(shanghai_tz)
|
||||
end_time = datetime.datetime.now(shanghai_tz)
|
||||
process_time = (end_time - start_time).total_seconds()
|
||||
job_id = event.job_id
|
||||
if "-temp-" in job_id:
|
||||
job_id = job_id.split("-")[0]
|
||||
# print("任务标识符:", event.job_id)
|
||||
# print("任务开始执行时间:", start_time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
# print("任务执行完成时间:", end_time.strftime("%Y-%m-%d %H:%M:%S"))
|
||||
# print("任务耗时(秒):", process_time)
|
||||
# print("任务返回值:", event.retval)
|
||||
# print("异常信息:", event.exception)
|
||||
# print("堆栈跟踪:", event.traceback)
|
||||
|
||||
result = {
|
||||
"job_id": job_id,
|
||||
"start_time": start_time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"end_time": end_time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"process_time": process_time,
|
||||
"retval": json.dumps(event.retval),
|
||||
"exception": json.dumps(event.exception),
|
||||
"traceback": json.dumps(event.traceback)
|
||||
}
|
||||
|
||||
db = get_database()
|
||||
try:
|
||||
task = db.get_data(SCHEDULER_TASK, job_id, is_object_id=True)
|
||||
result["job_class"] = task.get("job_class", None)
|
||||
result["name"] = task.get("name", None)
|
||||
result["group"] = task.get("group", None)
|
||||
result["exec_strategy"] = task.get("exec_strategy", None)
|
||||
result["expression"] = task.get("expression", None)
|
||||
except ValueError as e:
|
||||
result["exception"] = str(e)
|
||||
logger.error(f"任务编号:{event.job_id},报错:{e}")
|
||||
db.create_data(SCHEDULER_TASK_RECORD, result)
|
||||
|
30
kinit-task/core/logger.py
Normal file
30
kinit-task/core/logger.py
Normal file
@ -0,0 +1,30 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:10
|
||||
# @File : logger.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 日志管理器
|
||||
|
||||
import os
|
||||
import time
|
||||
from loguru import logger
|
||||
from application.settings import BASE_DIR
|
||||
|
||||
"""
|
||||
# 日志简单配置
|
||||
# 具体其他配置 可自行参考 https://github.com/Delgan/loguru
|
||||
"""
|
||||
|
||||
# 移除控制台输出
|
||||
logger.remove(handler_id=None)
|
||||
|
||||
log_path = os.path.join(BASE_DIR, 'logs')
|
||||
if not os.path.exists(log_path):
|
||||
os.mkdir(log_path)
|
||||
|
||||
log_path_info = os.path.join(log_path, f'info_{time.strftime("%Y-%m-%d")}.log')
|
||||
log_path_error = os.path.join(log_path, f'error_{time.strftime("%Y-%m-%d")}.log')
|
||||
|
||||
info = logger.add(log_path_info, rotation="00:00", retention="3 days", enqueue=True, encoding="UTF-8", level="INFO")
|
||||
error = logger.add(log_path_error, rotation="00:00", retention="3 days", enqueue=True, encoding="UTF-8", level="ERROR")
|
7
kinit-task/core/mongo/__init__.py
Normal file
7
kinit-task/core/mongo/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
from .mongo_manage import MongoManage
|
||||
|
||||
db = MongoManage()
|
||||
|
||||
|
||||
def get_database() -> MongoManage:
|
||||
return db
|
118
kinit-task/core/mongo/mongo_manage.py
Normal file
118
kinit-task/core/mongo/mongo_manage.py
Normal file
@ -0,0 +1,118 @@
|
||||
import datetime
|
||||
from typing import Any
|
||||
from bson import ObjectId
|
||||
from bson.errors import InvalidId
|
||||
from pymongo import MongoClient
|
||||
from pymongo.results import InsertOneResult, UpdateResult
|
||||
from pymongo.mongo_client import MongoClient as MongoClientType
|
||||
from pymongo.database import Database
|
||||
|
||||
|
||||
class MongoManage:
|
||||
"""
|
||||
mongodb 数据库管理器
|
||||
mongodb 官网:https://www.mongodb.com/docs/drivers/pymongo/
|
||||
"""
|
||||
|
||||
client: MongoClientType = None
|
||||
db: Database = None
|
||||
|
||||
def connect_to_database(self, path: str, db_name: str) -> None:
|
||||
"""
|
||||
连接 mongodb 数据库
|
||||
|
||||
:param path: mongodb 链接地址
|
||||
:param db_name: 数据库名称
|
||||
:return:
|
||||
"""
|
||||
self.client = MongoClient(path)
|
||||
self.db = self.client[db_name]
|
||||
|
||||
def close_database_connection(self) -> None:
|
||||
"""
|
||||
关闭 mongodb 数据库连接
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.client.close()
|
||||
|
||||
def create_data(self, collection: str, data: dict) -> InsertOneResult:
|
||||
"""
|
||||
创建单个数据
|
||||
|
||||
:param collection: 集合
|
||||
:param data: 数据
|
||||
"""
|
||||
data['create_datetime'] = datetime.datetime.now()
|
||||
data['update_datetime'] = datetime.datetime.now()
|
||||
result = self.db[collection].insert_one(data)
|
||||
# 判断插入是否成功
|
||||
if result.acknowledged:
|
||||
return result
|
||||
else:
|
||||
raise ValueError("创建新数据失败")
|
||||
|
||||
def get_data(
|
||||
self,
|
||||
collection: str,
|
||||
_id: str = None,
|
||||
v_return_none: bool = False,
|
||||
v_schema: Any = None,
|
||||
is_object_id: bool = False,
|
||||
**kwargs
|
||||
) -> dict | None:
|
||||
"""
|
||||
获取单个数据,默认使用 ID 查询,否则使用关键词查询
|
||||
|
||||
:param collection: 集合
|
||||
:param _id: 数据 ID
|
||||
:param v_return_none: 是否返回空 None,否则抛出异常,默认抛出异常
|
||||
:param is_object_id: 是否为 ObjectId
|
||||
:param v_schema: 指定使用的序列化对象
|
||||
"""
|
||||
if _id and is_object_id:
|
||||
kwargs["_id"] = ObjectId(_id)
|
||||
params = self.filter_condition(**kwargs)
|
||||
data = self.db[collection].find_one(params)
|
||||
if not data and v_return_none:
|
||||
return None
|
||||
elif not data:
|
||||
raise ValueError("查询单个数据失败,未找到匹配的数据")
|
||||
elif data and v_schema:
|
||||
return v_schema(**data).dict()
|
||||
return data
|
||||
|
||||
def put_data(self, collection: str, _id: str, data: dict, is_object_id: bool = False) -> UpdateResult:
|
||||
"""
|
||||
更新数据
|
||||
"""
|
||||
new_data = {'$set': data}
|
||||
result = self.db[collection].update_one({'_id': ObjectId(_id) if is_object_id else _id}, new_data)
|
||||
|
||||
if result.matched_count > 0:
|
||||
return result
|
||||
else:
|
||||
raise ValueError("更新数据失败,未找到匹配的数据")
|
||||
|
||||
@classmethod
|
||||
def filter_condition(cls, **kwargs) -> dict:
|
||||
"""
|
||||
过滤条件
|
||||
"""
|
||||
params = {}
|
||||
for k, v in kwargs.items():
|
||||
if not v:
|
||||
continue
|
||||
elif isinstance(v, tuple):
|
||||
if v[0] == "like" and v[1]:
|
||||
params[k] = {'$regex': v[1]}
|
||||
elif v[0] == "between" and len(v[1]) == 2:
|
||||
params[k] = {'$gte': f"{v[1][0]} 00:00:00", '$lt': f"{v[1][1]} 23:59:59"}
|
||||
elif v[0] == "ObjectId" and v[1]:
|
||||
try:
|
||||
params[k] = ObjectId(v[1])
|
||||
except InvalidId:
|
||||
raise ValueError("任务编号格式不正确!")
|
||||
else:
|
||||
params[k] = v
|
||||
return params
|
7
kinit-task/core/redis/__init__.py
Normal file
7
kinit-task/core/redis/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
from .redis_manage import RedisManage
|
||||
|
||||
db = RedisManage()
|
||||
|
||||
|
||||
def get_database() -> RedisManage:
|
||||
return db
|
37
kinit-task/core/redis/redis_manage.py
Normal file
37
kinit-task/core/redis/redis_manage.py
Normal file
@ -0,0 +1,37 @@
|
||||
import redis
|
||||
|
||||
|
||||
class RedisManage:
|
||||
"""
|
||||
redis 数据库管理器
|
||||
"""
|
||||
|
||||
rd: redis.Redis = None
|
||||
|
||||
def connect_to_database(self, path: str) -> None:
|
||||
"""
|
||||
连接 redis 数据库
|
||||
|
||||
:param path: mongodb 链接地址
|
||||
:return:
|
||||
"""
|
||||
self.rd = redis.from_url(path)
|
||||
|
||||
def close_database_connection(self) -> None:
|
||||
"""
|
||||
关闭 redis 连接
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.rd.close()
|
||||
|
||||
def subscribe(self, channel: str):
|
||||
"""
|
||||
订阅
|
||||
|
||||
:param channel: 频道
|
||||
:return:
|
||||
"""
|
||||
pubsub = self.rd.pubsub()
|
||||
pubsub.subscribe(channel)
|
||||
return pubsub
|
347
kinit-task/core/scheduler.py
Normal file
347
kinit-task/core/scheduler.py
Normal file
@ -0,0 +1,347 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:10
|
||||
# @File : scheduler.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
|
||||
import datetime
|
||||
import importlib
|
||||
from typing import List
|
||||
import re
|
||||
from apscheduler.jobstores.base import JobLookupError
|
||||
from apscheduler.jobstores.mongodb import MongoDBJobStore
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from apscheduler.triggers.date import DateTrigger
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
from apscheduler.job import Job
|
||||
from .listener import before_job_execution
|
||||
from apscheduler.events import EVENT_JOB_EXECUTED
|
||||
from application.settings import MONGO_DB_NAME, SCHEDULER_TASK_JOBS, TASKS_ROOT
|
||||
from core.mongo import get_database
|
||||
|
||||
|
||||
class Scheduler:
|
||||
TASK_DIR = TASKS_ROOT
|
||||
COLLECTION = SCHEDULER_TASK_JOBS
|
||||
|
||||
def __init__(self):
|
||||
self.scheduler = None
|
||||
self.db = None
|
||||
|
||||
def start(self, listener: bool = True) -> None:
|
||||
"""
|
||||
创建调度器
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.scheduler = BackgroundScheduler()
|
||||
if listener:
|
||||
# 注册事件监听器
|
||||
self.scheduler.add_listener(before_job_execution, EVENT_JOB_EXECUTED)
|
||||
self.scheduler.add_jobstore(self.__get_mongodb_job_store())
|
||||
self.scheduler.start()
|
||||
|
||||
def __get_mongodb_job_store(self) -> MongoDBJobStore:
|
||||
"""
|
||||
获取 MongoDBJobStore
|
||||
|
||||
:return: MongoDBJobStore
|
||||
"""
|
||||
self.db = get_database()
|
||||
return MongoDBJobStore(database=MONGO_DB_NAME, collection=self.COLLECTION, client=self.db.client)
|
||||
|
||||
def add_job(
|
||||
self,
|
||||
job_class: str,
|
||||
trigger: CronTrigger | DateTrigger | IntervalTrigger,
|
||||
name: str = None,
|
||||
*args,
|
||||
**kwargs
|
||||
) -> None | Job:
|
||||
"""
|
||||
date触发器用于在指定的日期和时间触发一次任务。它适用于需要在特定时间点执行一次的任务,例如执行一次备份操作。
|
||||
|
||||
:param job_class: 类路径
|
||||
:param trigger: 触发条件
|
||||
:param name: 任务名称
|
||||
:return:
|
||||
"""
|
||||
job_class = self.__import_module(job_class)
|
||||
if job_class:
|
||||
return self.scheduler.add_job(job_class.main, trigger=trigger, args=args, kwargs=kwargs, id=name)
|
||||
else:
|
||||
raise ValueError(f"添加任务失败,未找到该模块下的方法:{job_class}")
|
||||
|
||||
def add_cron_job(
|
||||
self,
|
||||
job_class: str,
|
||||
expression: str,
|
||||
start_date: str = None,
|
||||
end_date: str = None,
|
||||
timezone: str = "Asia/Shanghai",
|
||||
name: str = None,
|
||||
args: tuple = (),
|
||||
**kwargs
|
||||
) -> None | Job:
|
||||
"""
|
||||
通过 cron 表达式添加定时任务
|
||||
|
||||
:param job_class: 类路径
|
||||
:param expression: cron 表达式,六位或七位,分别表示秒、分钟、小时、天、月、星期几、年
|
||||
:param start_date: 触发器的开始日期时间。可选参数,默认为 None。
|
||||
:param end_date: 触发器的结束日期时间。可选参数,默认为 None。
|
||||
:param timezone: 时区,表示触发器应用的时区。可选参数,默认为 None,使用上海默认时区。
|
||||
:param name: 任务名称
|
||||
:param args: 非关键字参数
|
||||
:return:
|
||||
"""
|
||||
second, minute, hour, day, month, day_of_week, year = self.__parse_cron_expression(expression)
|
||||
trigger = CronTrigger(
|
||||
second=second,
|
||||
minute=minute,
|
||||
hour=hour,
|
||||
day=day,
|
||||
month=month,
|
||||
day_of_week=day_of_week,
|
||||
year=year,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
timezone=timezone
|
||||
)
|
||||
return self.add_job(job_class, trigger, name, *args, **kwargs)
|
||||
|
||||
def add_date_job(self, job_class: str, expression: str, name: str = None, args: tuple = (), **kwargs) -> None | Job:
|
||||
"""
|
||||
date触发器用于在指定的日期和时间触发一次任务。它适用于需要在特定时间点执行一次的任务,例如执行一次备份操作。
|
||||
|
||||
:param job_class: 类路径
|
||||
:param expression: date
|
||||
:param name: 任务名称
|
||||
:param args: 非关键字参数
|
||||
:return:
|
||||
"""
|
||||
trigger = DateTrigger(run_date=expression)
|
||||
return self.add_job(job_class, trigger, name, *args, **kwargs)
|
||||
|
||||
def add_interval_job(
|
||||
self,
|
||||
job_class: str,
|
||||
expression: str,
|
||||
start_date: str | datetime.datetime = None,
|
||||
end_date: str | datetime.datetime = None,
|
||||
timezone: str = "Asia/Shanghai",
|
||||
jitter: int = None,
|
||||
name: str = None,
|
||||
args: tuple = (),
|
||||
**kwargs
|
||||
) -> None | Job:
|
||||
"""
|
||||
date触发器用于在指定的日期和时间触发一次任务。它适用于需要在特定时间点执行一次的任务,例如执行一次备份操作。
|
||||
|
||||
:param job_class: 类路径
|
||||
:param expression:interval 表达式,分别为:秒、分、时、天、周,例如,设置 10 * * * * 表示每隔 10 秒执行一次任务。
|
||||
:param end_date: 表示任务的结束时间,可以设置为 datetime 对象或者字符串。
|
||||
例如,设置 end_date='2023-06-23 10:00:00' 表示任务在 2023 年 6 月 23 日 10 点结束。
|
||||
:param start_date: 表示任务的起始时间,可以设置为 datetime 对象或者字符串。
|
||||
例如,设置 start_date='2023-06-22 10:00:00' 表示从 2023 年 6 月 22 日 10 点开始执行任务。
|
||||
:param timezone:表示时区,可以设置为字符串或 pytz.timezone 对象。例如,设置 timezone='Asia/Shanghai' 表示使用上海时区。
|
||||
:param jitter:表示时间抖动,可以设置为整数或浮点数。例如,设置 jitter=2 表示任务的执行时间会在原定时间上随机增加 0~2 秒的时间抖动。
|
||||
:param name: 任务名称
|
||||
:param args: 非关键字参数
|
||||
:return:
|
||||
"""
|
||||
second, minute, hour, day, week = self.__parse_interval_expression(expression)
|
||||
trigger = IntervalTrigger(
|
||||
weeks=week,
|
||||
days=day,
|
||||
hours=hour,
|
||||
minutes=minute,
|
||||
seconds=second,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
timezone=timezone,
|
||||
jitter=jitter
|
||||
)
|
||||
return self.add_job(job_class, trigger, name, *args, **kwargs)
|
||||
|
||||
def run_job(self, job_class: str, args: tuple = (), **kwargs) -> None:
|
||||
"""
|
||||
立即执行一次任务,但不会执行监听器,只适合只需要执行任务,不需要记录的任务
|
||||
|
||||
:param job_class: 类路径
|
||||
:param args: 类路径
|
||||
:return: 类实例
|
||||
"""
|
||||
job_class = self.__import_module(job_class)
|
||||
job_class.main(*args, **kwargs)
|
||||
|
||||
def remove_job(self, name: str) -> None:
|
||||
"""
|
||||
删除任务
|
||||
|
||||
:param name: 任务名称
|
||||
:return:
|
||||
"""
|
||||
try:
|
||||
self.scheduler.remove_job(name)
|
||||
except JobLookupError as e:
|
||||
raise ValueError(f"删除任务失败, 报错:{e}")
|
||||
|
||||
def get_job(self, name: str) -> Job:
|
||||
"""
|
||||
获取任务
|
||||
|
||||
:param name: 任务名称
|
||||
:return:
|
||||
"""
|
||||
return self.scheduler.get_job(name)
|
||||
|
||||
def has_job(self, name: str) -> bool:
|
||||
"""
|
||||
判断任务是否存在
|
||||
|
||||
:param name: 任务名称
|
||||
:return:
|
||||
"""
|
||||
if self.get_job(name):
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
def get_jobs(self) -> List[Job]:
|
||||
"""
|
||||
获取所有任务
|
||||
|
||||
:return:
|
||||
"""
|
||||
return self.scheduler.get_jobs()
|
||||
|
||||
def get_job_names(self) -> List[str]:
|
||||
"""
|
||||
获取所有任务
|
||||
|
||||
:return:
|
||||
"""
|
||||
jobs = self.scheduler.get_jobs()
|
||||
return [job.id for job in jobs]
|
||||
|
||||
def __import_module(self, expression: str):
|
||||
"""
|
||||
反射模块
|
||||
|
||||
:param expression: 类路径
|
||||
:return: 类实例
|
||||
"""
|
||||
module, args = self.__parse_string_to_class(expression)
|
||||
module_pag = self.TASK_DIR + '.' + module[0:module.rindex(".")]
|
||||
module_class = module[module.rindex(".") + 1:]
|
||||
try:
|
||||
# 动态导入模块
|
||||
pag = importlib.import_module(module_pag)
|
||||
return getattr(pag, module_class)(*args)
|
||||
except ModuleNotFoundError:
|
||||
raise ValueError(f"未找到该模块:{module_pag}")
|
||||
except AttributeError:
|
||||
raise ValueError(f"未找到该模块下的方法:{module_class}")
|
||||
except TypeError as e:
|
||||
raise ValueError(f"参数传递错误:{args}, 详情:{e}")
|
||||
|
||||
@staticmethod
|
||||
def __parse_cron_expression(expression: str) -> tuple:
|
||||
"""
|
||||
解析 cron 表达式
|
||||
|
||||
:param expression: cron 表达式,支持六位或七位,分别表示秒、分钟、小时、天、月、星期几、年
|
||||
:return: 解析后的秒、分钟、小时、天、月、星期几、年字段的元组
|
||||
"""
|
||||
fields = expression.strip().split()
|
||||
|
||||
if len(fields) not in (6, 7):
|
||||
raise ValueError("无效的 Cron 表达式")
|
||||
|
||||
parsed_fields = [None if field in ('*', '?') else field for field in fields]
|
||||
if len(fields) == 6:
|
||||
parsed_fields.append(None)
|
||||
|
||||
return tuple(parsed_fields)
|
||||
|
||||
@staticmethod
|
||||
def __parse_interval_expression(expression: str) -> tuple:
|
||||
"""
|
||||
解析 interval 表达式
|
||||
|
||||
:param expression: interval 表达式,分别为:秒、分、时、天、周,例如,设置 10 * * * * 表示每隔 10 秒执行一次任务。
|
||||
:return:
|
||||
"""
|
||||
# 将传入的 interval 表达式拆分为不同的字段
|
||||
fields = expression.strip().split()
|
||||
|
||||
if len(fields) != 5:
|
||||
raise ValueError("无效的 interval 表达式")
|
||||
|
||||
parsed_fields = [int(field) if field != '*' else 0 for field in fields]
|
||||
return tuple(parsed_fields)
|
||||
|
||||
@classmethod
|
||||
def __parse_string_to_class(cls, expression: str) -> tuple:
|
||||
"""
|
||||
使用正则表达式匹配类路径和参数
|
||||
|
||||
:param expression: 表达式
|
||||
:return:
|
||||
"""
|
||||
pattern = r'([\w.]+)(?:\((.*)\))?'
|
||||
match = re.match(pattern, expression)
|
||||
|
||||
if match:
|
||||
class_path = match.group(1)
|
||||
arguments = match.group(2)
|
||||
|
||||
if arguments:
|
||||
arguments = cls.__parse_arguments(arguments)
|
||||
else:
|
||||
arguments = []
|
||||
|
||||
return class_path, arguments
|
||||
|
||||
return None, None
|
||||
|
||||
@staticmethod
|
||||
def __parse_arguments(args_str) -> list:
|
||||
"""
|
||||
解析类路径参数字符串
|
||||
|
||||
:param args_str: 类参数字符串
|
||||
:return:
|
||||
"""
|
||||
arguments = []
|
||||
|
||||
for arg in re.findall(r'"([^"]*)"|(\d+\.\d+)|(\d+)|([Tt]rue|[Ff]alse)', args_str):
|
||||
if arg[0]:
|
||||
# 字符串参数
|
||||
arguments.append(arg[0])
|
||||
elif arg[1]:
|
||||
# 浮点数参数
|
||||
arguments.append(float(arg[1]))
|
||||
elif arg[2]:
|
||||
# 整数参数
|
||||
arguments.append(int(arg[2]))
|
||||
elif arg[3]:
|
||||
# 布尔参数
|
||||
if arg[3].lower() == 'true':
|
||||
arguments.append(True)
|
||||
else:
|
||||
arguments.append(False)
|
||||
|
||||
return arguments
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""
|
||||
关闭调度器
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.scheduler.shutdown()
|
0
kinit-task/logs/.gitkeep
Normal file
0
kinit-task/logs/.gitkeep
Normal file
164
kinit-task/main.py
Normal file
164
kinit-task/main.py
Normal file
@ -0,0 +1,164 @@
|
||||
import atexit
|
||||
import datetime
|
||||
import json
|
||||
import random
|
||||
from enum import Enum
|
||||
from apscheduler.jobstores.base import ConflictingIdError
|
||||
from core.scheduler import Scheduler
|
||||
from core.mongo import get_database as get_mongo
|
||||
from application.settings import MONGO_DB_NAME, MONGO_DB_URL, REDIS_DB_URL, SUBSCRIBE, SCHEDULER_TASK, \
|
||||
SCHEDULER_TASK_RECORD
|
||||
from core.redis import get_database as get_redis
|
||||
from core.logger import logger
|
||||
|
||||
|
||||
class ScheduledTask:
|
||||
|
||||
class JobExecStrategy(Enum):
|
||||
interval = "interval"
|
||||
date = "date"
|
||||
cron = "cron"
|
||||
once = "once"
|
||||
|
||||
def __init__(self):
|
||||
self.mongo = None
|
||||
self.scheduler = None
|
||||
self.rd = None
|
||||
|
||||
def add_job(self, exec_strategy: str, job_params: dict) -> None:
|
||||
"""
|
||||
添加定时任务
|
||||
|
||||
:param exec_strategy: 执行策略
|
||||
:param job_params: 执行参数
|
||||
:return:
|
||||
"""
|
||||
name = job_params.get("name", None)
|
||||
error_info = None
|
||||
try:
|
||||
if exec_strategy == self.JobExecStrategy.interval.value:
|
||||
self.scheduler.add_interval_job(**job_params)
|
||||
elif exec_strategy == self.JobExecStrategy.cron.value:
|
||||
self.scheduler.add_cron_job(**job_params)
|
||||
elif exec_strategy == self.JobExecStrategy.date.value:
|
||||
self.scheduler.add_date_job(**job_params)
|
||||
elif exec_strategy == self.JobExecStrategy.once.value:
|
||||
# 这种方式会自动执行事件监听器,用于保存执行任务完成后的日志
|
||||
job_params["name"] = f"{name}-temp-{random.randint(1000, 9999)}"
|
||||
self.scheduler.add_date_job(**job_params, expression=datetime.datetime.now())
|
||||
else:
|
||||
raise ValueError("无效的触发器")
|
||||
except ConflictingIdError as e:
|
||||
# 任务编号已存在,重复添加报错
|
||||
error_info = "任务编号已存在"
|
||||
except ValueError as e:
|
||||
error_info = e.__str__()
|
||||
|
||||
if error_info:
|
||||
logger.error(f"任务编号:{name},报错:{error_info}")
|
||||
self.error_record(name, error_info)
|
||||
|
||||
def error_record(self, name: str, error_info: str) -> None:
|
||||
"""
|
||||
添加任务失败记录,并且将任务状态改为 False
|
||||
|
||||
:param name: 任务编号
|
||||
:param error_info: 报错信息
|
||||
:return:
|
||||
"""
|
||||
try:
|
||||
self.mongo.put_data(SCHEDULER_TASK, name, {"is_active": False})
|
||||
task = self.mongo.get_data(SCHEDULER_TASK, name)
|
||||
# 执行你想要在任务执行前执行的代码
|
||||
result = {
|
||||
"job_id": name,
|
||||
"job_class": task.get("job_class", None),
|
||||
"name": task.get("name", None),
|
||||
"group": task.get("group", None),
|
||||
"exec_strategy": task.get("exec_strategy", None),
|
||||
"expression": task.get("expression", None),
|
||||
"start_time": datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"end_time": datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"process_time": 0,
|
||||
"retval": "任务添加失败",
|
||||
"exception": error_info,
|
||||
"traceback": None
|
||||
}
|
||||
self.mongo.create_data(SCHEDULER_TASK_RECORD, result)
|
||||
except ValueError as e:
|
||||
logger.error(f"任务编号:{name}, 报错:{e}")
|
||||
|
||||
def run(self) -> None:
|
||||
"""
|
||||
启动监听订阅消息(阻塞)
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.start_mongo()
|
||||
self.start_scheduler()
|
||||
self.start_redis()
|
||||
|
||||
pubsub = self.rd.subscribe(SUBSCRIBE)
|
||||
|
||||
logger.info("已成功启动程序,等待接收消息...")
|
||||
print("已成功启动程序,等待接收消息...")
|
||||
|
||||
# 处理接收到的消息
|
||||
for message in pubsub.listen():
|
||||
if message['type'] == 'message':
|
||||
data = json.loads(message['data'].decode('utf-8'))
|
||||
operation = data.get("operation")
|
||||
task = data.get("task")
|
||||
content = f"接收到任务:任务操作方式({operation}),任务详情:{task}"
|
||||
logger.info(content)
|
||||
print(content)
|
||||
getattr(self, operation)(**task)
|
||||
|
||||
def start_mongo(self) -> None:
|
||||
"""
|
||||
启动 mongo
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.mongo = get_mongo()
|
||||
self.mongo.connect_to_database(MONGO_DB_URL, MONGO_DB_NAME)
|
||||
print("成功连接 MongoDB")
|
||||
|
||||
def start_scheduler(self) -> None:
|
||||
"""
|
||||
启动定时任务
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.scheduler = Scheduler()
|
||||
self.scheduler.start()
|
||||
print("成功启动 Scheduler")
|
||||
|
||||
def start_redis(self) -> None:
|
||||
"""
|
||||
启动 redis
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.rd = get_redis()
|
||||
self.rd.connect_to_database(REDIS_DB_URL)
|
||||
print("成功连接 Redis")
|
||||
|
||||
def close(self) -> None:
|
||||
"""
|
||||
# pycharm 执行停止,该函数无法正常被执行,怀疑是因为阻塞导致或 pycharm 的强制退出导致
|
||||
# 报错导致得退出,会被执行
|
||||
关闭程序
|
||||
|
||||
:return:
|
||||
"""
|
||||
self.mongo.close_database_connection()
|
||||
self.scheduler.shutdown()
|
||||
self.rd.close_database_connection()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main = ScheduledTask()
|
||||
atexit.register(main.close)
|
||||
main.run()
|
||||
|
11
kinit-task/requirements.txt
Normal file
11
kinit-task/requirements.txt
Normal file
@ -0,0 +1,11 @@
|
||||
APScheduler==3.10.1
|
||||
colorama==0.4.6
|
||||
dnspython==2.3.0
|
||||
loguru==0.7.0
|
||||
pymongo==4.3.3
|
||||
pytz==2023.3
|
||||
six==1.16.0
|
||||
tzdata==2023.3
|
||||
tzlocal==5.0.1
|
||||
win32-setctime==1.1.0
|
||||
redis==4.5.5
|
7
kinit-task/tasks/__init__.py
Normal file
7
kinit-task/tasks/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:07
|
||||
# @File : __init__.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
7
kinit-task/tasks/test/__init__.py
Normal file
7
kinit-task/tasks/test/__init__.py
Normal file
@ -0,0 +1,7 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:08
|
||||
# @File : __init__.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
26
kinit-task/tasks/test/main.py
Normal file
26
kinit-task/tasks/test/main.py
Normal file
@ -0,0 +1,26 @@
|
||||
#!/usr/bin/python
|
||||
# -*- coding: utf-8 -*-
|
||||
# @version : 1.0
|
||||
# @Create Time : 2023/6/21 10:08
|
||||
# @File : mian.py
|
||||
# @IDE : PyCharm
|
||||
# @desc : 简要说明
|
||||
import datetime
|
||||
import time
|
||||
|
||||
|
||||
class Test:
|
||||
|
||||
def __init__(self, name: str, age: int):
|
||||
self.name = name
|
||||
self.age = age
|
||||
|
||||
def main(self) -> str:
|
||||
"""
|
||||
主入口函数
|
||||
|
||||
:return:
|
||||
"""
|
||||
print('{}, 定时任务测试实例,参数为: {}, {}'.format(datetime.datetime.now(), self.name, self.age))
|
||||
time.sleep(3)
|
||||
return '任务执行完成'
|
Loading…
x
Reference in New Issue
Block a user