Fix#880
Design:
1. A global setting `security.TWO_FACTOR_AUTH`.
* To support org-level config, we need to introduce a better "owner
setting" system first (in the future)
2. A user without 2FA can login and may explore, but can NOT read or
write to any repositories via API/web.
3. Keep things as simple as possible.
* This option only aggressively suggest users to enable their 2FA at the
moment, it does NOT guarantee that users must have 2FA before all other
operations, it should be good enough for real world use cases.
* Some details and tests could be improved in the future since this
change only adds a check and seems won't affect too much.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
`[repository.pull-request] DELAY_CHECK_FOR_INACTIVE_DAYS` is a new
setting to delay the mergeable check for pull requests that have been
inactive for the specified number of days.
This avoids potentially long delays for big repositories with many pull
requests. and reduces system load overall when there are many
repositories or pull requests.
When viewing the PR, checking will start immediately and the PR merge
box will automatically reload when complete. Accessing the PR through
the API will also start checking immediately.
The default value of `7` provides a balance between system load, and
keeping behavior similar to what it was before both for users and API
access. With `0` all conflict checking will be delayed, while `-1`
always checks immediately to restore the previous behavior.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This pull request adds a GitHub-compatible API endpoint to lock and
unlock an issue.
The following routes exist now:
- `PUT /api/v1/repos/{owner}/{repo}/issues/{id}/lock` to lock an issue
- `DELETE /api/v1/repos/{owner}/{repo}/issues/{id}/lock` to unlock an issue
Fixes#33677Fixes#20012
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Implements runner apis based on
https://docs.github.com/en/rest/actions/self-hosted-runners?apiVersion=2022-11-28#list-self-hosted-runners-for-an-organization
- Add Post endpoints for registration-token, google/go-github revealed
this as problem
- We should deprecate Get Endpoints, leaving them for compatibility
- Get endpoint of admin has api path /admin/runners/registration-token
that feels wrong, /admin/actions/runners/registration-token seems more
consistent with user/org/repo api
- Get Runner Api
- List Runner Api
- Delete Runner Api
- Tests admin / user / org / repo level endpoints
Related to #33750 (implements point 1 and 2)
Via needs discovered in #32461, this runner api is needed to allow
cleanup of runners that are deallocated without user interaction.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Using existing "content" variable in `swift.go`
2. Do not report 500 server error in `GetPullDiffStats` middleware,
otherwise a PR missing ref won't be able to view.
3. Fix the abused "label button" when listing commits, there was too
much padding space, see the screenshot below.
This adds a middleware for overload protection that is intended to help protect against malicious scrapers.
It does this via [`codel`](https://github.com/bohde/codel), which will perform the following:
1. Limit the number of in-flight requests to some user-defined max
2. When in-flight requests have reached their begin queuing requests.
Logged-in requests having priority above logged-out requests
3. Once a request has been queued for too long,
it has a probabilistic chance to be rejected based on how overloaded the entire system is.
When a server experiences more traffic than it can handle,
this keeps latency low for logged-in users and rejects just
enough requests from logged-out users to not overload the service.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Fix#34188
The name "FileName" is ambiguous: sometimes it is "base name without
path", sometimes it is "full name with path".
The ambiguous name causes various problems.
This PR clarifies the usage: `FileTreePath`: the full name with path.
This PR moved git attributes related code to `modules/git/attribute` sub
package and moved language stats related code to
`modules/git/languagestats` sub package to make it easier to maintain.
And it also introduced a performance improvement which use the `git
check-attr --source` which can be run in a bare git repository so that
we don't need to create a git index file. The new parameter need a git
version >= 2.40 . If git version less than 2.40, it will fall back to
previous implementation.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: yp05327 <576951401@qq.com>
Fix#2616
This PR adds a new sort option for exclusive labels.
For exclusive labels, a new property is exposed called "order", while in
the UI options are populated automatically in the `Sort` column (see
screenshot below) for each exclusive label scope.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR uniform all temporary directory usage so that it will be easier
to manage.
Relate to #31792
- [x] Added a new setting to allow users to configure the global
temporary directory.
- [x] Move all temporary files and directories to be placed under
os.Temp()/gitea.
- [x] `setting.Repository.Local.LocalCopyPath` now will be
`setting.TempPath/local-repo` and the customized path is removed.
```diff
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;[repository.local]
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-;;
-;; Path for local repository copy. Defaults to TEMP_PATH + `local-repo`, this is deprecated and cannot be changed
-;LOCAL_COPY_PATH = local-repo
```
- [x] `setting.Repository.Upload.TempPath` now will be
`settting.TempPath/uploads` and the customized path is removed.
```diff
;[repository.upload]
-;;
-;; Path for uploads. Defaults to TEMP_PATH + `uploads`
-;TEMP_PATH = uploads
```
- [x] `setting.Packages.ChunkedUploadPath` now will be
`settting.TempPath/package-upload` and the customized path is removed.
```diff
;[packages]
-;;
-;; Path for chunked uploads. Defaults it's `package-upload` under `TEMP_PATH` unless it's an absolute path.
-;CHUNKED_UPLOAD_PATH = package-upload
```
- [x] `setting.SSH.KeyTestPath` now will be
`settting.TempPath/ssh_key_test` and the customized path is removed.
```diff
[server]
-;;
-;; Directory to create temporary files in when testing public keys using ssh-keygen,
-;; default is the system temporary directory.
-;SSH_KEY_TEST_PATH =
```
TODO:
- [ ] setting.PprofDataPath haven't been changed because it may need to
be kept until somebody read it but temp path may be clean up any time.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This PR tries to finally fix the bug mentioned in #30011 and #15504,
where the user repo limit is checked when creating a repo in an
organization.
Fix#30011
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>
Extract from #33934
In the same goroutine, we should reuse the exist cat file sub process
which exist in `git.Repository` to avoid creating a unnecessary
temporary subprocess.
This PR reuse the exist cate file writer and reader in
`getCommitFromBatchReader`.
It also move `prepareLatestCommitInfo` before creating dataRc which will
hold the writer so other git operation will create a temporary cat file
subprocess.
Adds the `flat-square` style to action badges. Styles can be selected by
adding `?style=<style>` to the badge endpoint. If no style query is
given, or if the query is invalid, the style defaults to `flat`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#33966
```
;; User must sign in to view anything.
;; It could be set to "expensive" to block anonymous users accessing some pages which consume a lot of resources,
;; for example: block anonymous AI crawlers from accessing repo code pages.
;; The "expensive" mode is experimental and subject to change.
;REQUIRE_SIGNIN_VIEW = false
```
Follow #33127Fix#8649, fix#639
This is a complete solution. A repo unit could be set to:
* Anonymous read (non-signed-in user)
* Everyone read (signed-in user)
* Everyone write (wiki-only)
When a team have no code unit permission of a repository, the member of
the team should not view activity contributors, recent commits and code
frequrency.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The current action status badge are looking different from most other
badges renders, which is especially noticeable when using them along
with other badges. This PR updates the action badges to match the
commonly used badges from other providers.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Follow #33127
This PR add backend logic and test for "anonymous access", it shares the
same logic as "everyone access", so not too much change.
By the way, split `SettingsPost` into small functions to make it easier
to make frontend-related changes in the future.
Next PR will add frontend support for "anonymous access"
fixes an issue where user is unable to create new repository in
organization via UI if repository limits are in place and user has
exhausted them for their own namespace.
closes: https://github.com/go-gitea/gitea/issues/15504
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The pagination on the user dashboard sounds unnecessary, this will
change it to a prev/next buttons. For instances with around `10 million`
records in the action table, this option affects how the user dashboard
is loaded on first visit.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Two SQLs are very slow when `action` table have over 5M records.
```
database duration=1.8881s db.sql="SELECT created_unix DIV 900 * 900 AS timestamp, count(user_id) as contributions FROM `action` WHERE user_id=? AND act_user_id=? AND (created_unix > ?) GROUP BY timestamp ORDER BY timestamp"
database duration=1.5408s db.sql="SELECT count(*) FROM `action` WHERE (user_id = ?) AND (is_deleted = ?)"
```
This will cache the count for the first loading or when the activities
changed.
Currently the organisation feed only includes items for public
repositories (for non-administrators). This pull requests adds
notifications from private repositories to the organisation-feed (for
accounts that have access to the organisation).
Feed-items only get shown for repositories where the users team(s)
should have access to, this filtering seems to get done by some existing
code.
Needs some tests, but am unsure where/how to add them.
Before:

After:

---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Resolve#29328
This pull request introduces a file tree on the left side when reviewing
files of a repository.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#24623
Major changes:
1. Redirect `/owner/repo/blob/*` requests to `/owner/repo/src/commit/*`
(like GitHub)
2. Add a "view file diff" link (see screenshot below)
3. Refactor "AssertHTMLElement" to generic, now we can accurately assert
existence or number.
4. Add more tests
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: delvh <dev.lh@web.de>
Fix#33490
It will only read the changed file on the pushed commits but not all the
files of this PR.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fix#33358, fix#21970
This adds a step in the `GitDiffForRender` that does syntax highlighting for the
entire file and then only references lines from that syntax highlighted
code. This allows things like multi-line comments to be syntax
highlighted correctly.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The diff stats are no longer part of the diff generation.
Use `GetDiffShortStat` instead to get the total number of changed files,
added lines, and deleted lines.
As such, `gitdiff.GetDiff` can be simplified:
It should not do more than expected.
And do not run "git diff --shortstat" for pull list. Fix#31492
Modify Diff View FileTree to show all files
## Changes
* removes Show Status button on diff
* uses `git diff-tree` to generate the file tree for the diff
* doesn't reload the diff tree each time we load more files in the
preview
* selecting and unloaded file will keep loading until that file is
loaded
* removes `DiffFileList.vue` and "Show Stats" in diff options
## Open Questions
* selecting and unloaded file will keep loading until that file is
loaded. Is this behaviour okay? It matches what github does.
### Demo
In this demo I set `git.MAX_GIT_DIFF_FILES=1` in my `app.ini` to
demonstrate a worst case example. In most cases the behaviour isn't
nearly as jarring as we load a bunch of files at a time.
https://github.com/user-attachments/assets/72f29663-d6fc-472d-94fa-7fb5950c2836
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
See the FIXME comment in code. Otherwise, if a repo's issue unit is
disabled, then the PRs can't be edited anymore.
By the way, make the permission log output look slightly better.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: metiftikci <metiftikci@hotmail.com>
Noticed a SQL in gitea.com has a bigger load. It seems both `is_pull`
and `pin_order` are not indexed columns in the database.
```SQL
SELECT `id`, `repo_id`, `index`, `poster_id`, `original_author`, `original_author_id`, `name`, `content`, `content_version`, `milestone_id`, `priority`, `is_closed`, `is_pull`, `num_comments`, `ref`, `pin_order`, `deadline_unix`, `created_unix`, `updated_unix`, `closed_unix`, `is_locked`, `time_estimate` FROM `issue` WHERE (repo_id =?) AND (is_pull = 0) AND (pin_order > 0) ORDER BY pin_order
```
I came across a comment
https://github.com/go-gitea/gitea/pull/24406#issuecomment-1527747296
from @delvh , which presents a more reasonable approach. Based on this,
this PR will migrate all issue and pull request pin data from the
`issue` table to the `issue_pin` table. This change benefits larger
Gitea instances by improving scalability and performance.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* download endpoint has to use 302 redirect
* fake blob download used if direct download not possible
* downloading v3 artifacts not possible
New repo apis based on GitHub Rest V3
- GET /runs/{run}/artifacts (Cannot use run index of url due to not
being unique)
- GET /artifacts
- GET + DELETE /artifacts/{artifact_id}
- GET /artifacts/{artifact_id}/zip
- (GET /artifacts/{artifact_id}/zip/raw this is a workaround for a http
302 assertion in actions/toolkit)
- api docs removed this is protected by a signed url like the internal
artifacts api and no longer usable with any token or swagger
- returns http 401 if the signature is invalid
- or change the artifact id
- or expired after 1 hour
Closes#33353Closes#32124
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
When listing commits, Gitea attempts to retrieve the actual user based
on the commit email. Querying users one by one from the database is
inefficient. This PR optimizes the process by batch querying users by
email, reducing the number of database queries.
Some old code use direct type-casting to get context, it causes
problems.
This PR fixes all legacy problems and use correct `ctx.Value` to get
low-level contexts.
Fix#33518
- Find the runner before deleting
- Move the main logic from `routers/web/repo/setting/runners.go` to
`routers/web/shared/actions/runners.go`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- Find the variable before updating or deleting
- Move the main logic from `routers/web/repo/setting/variables.go` to
`routers/web/shared/actions/variables.go`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
This PR fixes#33205.
If stars are disabled:
* The `.../repo/stars` page returns a 403 Forbidden error
* Star-related API endpoints return a 403 Forbidden error saying `Stars are disabled.`
* Same for action endpoints
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. Fix incorrect `MentionCount` (actually it seems to be deadcode,
affects nothing)
2. Remove fallback sha1 support for time limit token
3. Use route middleware `reqRepoActionsWriter` for `ArtifactsDeleteView`
4. Use clearer message "Failed to authenticate user" instead of "Verify"
when auth fails
5. `tests/integration/benchmarks_test.go` is not quite right, actually
it is never used, so delete it.
6. Remove or update TODO comments
Dear Gitea team,
first of all, thanks for the great work you're doing with this project.
I'm planning to introduce Gitea at a client site, and noticed that while
there is time recording, there are no project-manager-friendly reports
to actually make use of that data, as were also mentioned by others in
#4870#8684 and #13531.
Since I had a little time last weekend, I had put together something
that I hope to be a useful contribution to this great project (while of
course useful for me too).
This PR adds a new "Worktime" tab to the Organisation level. There is a
date range selector (by default set to the current month), and there are
three possible views:
- by repository,
- by milestone, and
- by team member.
Happy to receive any feedback!
There are several possible future improvements of course (predefined
date ranges, charts, a member time sheet, matrix of repos/members, etc)
but I hope that even in this relatively simple state this would be
useful to lots of people.
<img width="1161" alt="Screen Shot 2022-05-25 at 22 12 58"
src="https://user-images.githubusercontent.com/118010/170366976-af00c7af-c4f3-4117-86d7-00356d6797a5.png">
Keep up the good work!
Kristof
---------
Co-authored-by: user <user@kk-git1>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
- Both have `RejectTransfer` and `CancelTransfer` because the permission
checks are not the same. `CancelTransfer` can be done by the doer or
those who have admin permission to access this repository.
`RejectTransfer` can be done by the receiver user if it's an individual
or those who can create repositories if it's an organization.
- Some tests are wrong, this PR corrects them.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This is a follow-up to https://github.com/go-gitea/gitea/pull/33097.
When linking a submodule at a commit in either the repo view, or a diff
when adding a new submodule, link to the tree view of that submodules
intead of the individual commit. This shows the user the full tree,
instead of the diff of the commit.
This makes the assumption that the tree for a given SHA is at
`<repo_url>/tree/<sha>`. This URL format is supported by both Github &
Gitlab, but not Gitea. To fix this, add a redirect from
`<username>/<repo>/tree/<ref>` to `<username>/<repo>/src/<ref>`, so that
Gitea can support this URL structure.
1. add `GetSiteCookieFlashMessage` to help to parse flash message
2. clarify `handleRepoHomeFeed` logic
3. remove unnecessary global variables, use `sync.OnceValue` instead
4. add some tests for `IsUsableUsername` and `IsUsableRepoName`
Extract from #33320
This PR uses a map instead of a struct to store webhook event
information. It removes many duplicated functions and makes the logic
clearer.
1. Add a OpenTelemetry-like shim-layer to collect traces
2. Add a simple builtin trace collector and exporter, end users could
download the diagnosis report to get the traces.
This PR's design is quite lightweight, no hard-dependency, and it is
easy to improve or remove. We can try it on gitea.com first to see
whether it works well, and fine tune the details.
---------
Co-authored-by: silverwind <me@silverwind.io>
After the RefName refactoring, the `ctx.Repo.CommitID` is only set when
there is a `RepoRefByType` middleware.
Many handlers do not use that middleware and they only use "default
branch"
Fix password form missing whilst linking account even with
`ENABLE_PASSWORD_SIGNIN_FORM = true`.
Remove redundant empty box in account linking sign up page when
`LinkAccountMode` is true.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Estimated time represented in hours it might be convenient to
have tracked time represented in the same way to be compared and
managed.
---------
Co-authored-by: Sysoev, Vladimir <i@vsysoev.ru>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. the error check was added for go-git, it was caused by the empty `.keep` file in the test fixtures.
2. use `PostFormValue` instead of `PostForm.Get` (`Get` needs to parse the form ahead)
3. fix incorrect container text (it should show "Images" in the header but not "OS/Arch")
4. align maven xml
5. fix closed PR color&icon
Currently, anyone with write permissions to a repo are able to rename
default or protected branches.
This change follows
[GitHub's](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-branches-in-your-repository/renaming-a-branch)
design by only allowing repo/site admins to change these branches.
However, it also follows are current design for protected branches and
only allows admins to modify branch names == branch protection rule
names. Glob-based rules cannot be renamed by anyone (as was already the
case, but we now catch `ErrBranchIsProtected` which we previously did
not catch, throwing a 500).
Major changes:
1. do not sync ".keep" file during tests
2. fix incorrect route handler and empty repo handling (backported as #33253 with tests)
3. do not use `RepoRef`: most of the calls are abuses.
4. Use `git.RefType` instead of a new type definition `RepoRefType` on `context`.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Found while investigating #33210.
This line no longer makes sense because the form field "TagName" is
required, so this would mean that this code path would never be covered.
Because it isn't covered, we end up going down the "update release"
logic where we eventually set `Release.IsTag` to false (meaning it will
now be treated as a release instead of a tag).
This snapshot rewrites the condition to ensure that we aren't trying to
create a tag that already exists.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
1. "blob" is not a "ref", it shouldn't (and not unable to) be handled by
`RepoRefByType`
2. the `/blob/{sha}` handle should use the path param "sha" directly
The `ctx.Repo.RefName` was used to be a "short name", it causes a lot of
ambiguity.
This PR does some refactoring and use `RefFullName` to replace the
legacy `RefName`, and simplify RepoAssignment
Resolve#32341
~Depends on #27151~
- [x] It will display a checkbox of deleting the head branch on the pull
request view page when starting an auto-merge task.
- [x] Add permission check before deleting the branch
- [x] Add delete branch comment for those closing pull requests because
of head branch or base branch was deleted.
- [x] Merge `RetargetChildrenOnMerge` and `AddDeletePRBranchComment`
into `service.DeleteBranch`.
This adds links to submodules in diffs, similar to the existing link
when viewing a repo at a specific commit. It does this by expanding diff
parsing to recognize changes to submodules, and find the specific refs
that are added, deleted or changed.
Related #25888
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
close#33086
* Add a special value for "SSH_USER" setting: `(DOER_USERNAME)`
* Improve parseRepositoryURL and add tests (now it doesn't have hard
dependency on some setting values)
Many changes are just adding "ctx" and "doer" argument to functions.
By the way, improve app.example.ini, remove all `%(key)s` syntax, it
only makes messy and no user really cares about it.
Document: https://gitea.com/gitea/docs/pulls/138
This PR fixes a performance problem when reviewing a pull request in a
big instance which have many records in the `review` table.
Traditionally, we should add more indexes in that table. But since
dismissed reviews of 1 pull request will not be too many as expected in
a common repository. Filtering reviews in the memory should be more
quick .
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
fixes#33119
routers/web/repo/view_home.go

Calling `updateContextRepoEmptyAndStatus` will always ask the DB to
update the updated Unix attributes.
When revisiting the repo's home page, the timestamp will be updated
unexpectedly, so I added the needsUpdate variable to check whether, in
the end, the commitment to db update is necessary if columns have not
changed at all.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
* htmx updates the input form on branch switch
* add workflow warning to dispatch modal
* use name if description of input is empty
* show error if workflow_dispatch not available on branch
Closes#33073Closes#33099
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The latest commit cache is currently used only for listing tree files.
However, a cold start may take longer than directly invoking the Git
command. This PR addresses the issue of slow response times when
accessing raw files, improving performance in such scenarios.
```log
gitea.log:105521:2024/12/23 08:22:18 ...eb/routing/logger.go:68:func1() [W] router: slow GET /xxxx/xxxxxx/raw/commit/xxxxxxxxxxxxxxxxxxxxxxxxxxx/.editorconfig for 172.18.0.5:53252, elapsed 3526.8ms @ repo/download.go:117(repo.SingleDownload)
```