Celery – schedule tasks with pauses between them

I needed to schedule 10 tasks that have these conditions

  1. A time gap between tasks is fixed (but varies)
  2. No need to pass results of one task to another
  3. Need to read a database in every task (important: at the moment of the execution)

I tried a few options, and this one worked

 ret = chain(task1.si(args).set(countdown=30*1), ..., task10.si(args).set(countdown=30*10)).apply_sync()

Important note – the signature here is not .s() but .si() this way you don’t pass the result of task1 to task2. You set the countdown with set() and not inside the apply_sync().

It is simple but it took me some time to figure out.

To my from now on wandering friends (RU)

Сложные времена лишений впереди, но они как ничто другое толкают нас на великие свершения. Вы не одиноки. Всё когда-то кончается - и плохое, и хорошее.

Если кто-нибудь из ваших знакомых хочет заняться кодингом, и когда если не сейчас - пишите в телегу - я помогаю начинающим (и продолжающим) бесплатно.

A message for my russian-speaking readers:

Difficult times are ahead of us, but they are forcing us to do great things like nothing else. You are not alome. Everything ends - both bad and good.

If your friends are willing to try and start coding now for the first time, now or never - message my Telegram account - I am helping beginners who want to code.

What I am up to these unsettling days

The world is crazy, but I don’t let it deter me. I am a great believer in doing things during challenging times. So my brother and I are spending our spare (from work) time at Codewars and Hackerrank.

HackerRank

First, I returned to HackerRank to do some SQL, and I completed everything I found relevant to my current studies. It seems like from the lack of use I lost some of my SQL knowledge and I want to fix that.

My modest achievements at HackerRank

I plan to do much more with Problem Solving (there are a few data structures and algorithms I am not yet very familiar with). Maybe I will return and do more of Python, but at this point I just feel bored and uninspired.

SQL Academy

Then I jumped to SQL Academy, and I loved it, but it has too many bugs to my taste

UPD. 24.03.22 – I am a moron. I don’t see bugs, I just missed some details and therefore couldn’t successfully submit a few tasks. Now I bought a Premium account for $10 and realized my mistakes. I use it after I complete the tasks to see how a DB expert would solve it. Can’t wait to finish the task list by end of the week (wish me luck hehe).

One downside for me – it requires MySQL syntax, and I am not really interested in it. I personally find PostgreSQL more accomodating to Python users and those who don’t specialize solely in databases too.

The interface is fantastic and I love how you can see queries and tables and an ERM. I will be completing everything that bugs will let me – it has been very useful to me.

Codewars

Meanwhile my brother has been devoured by Codewars, so after all these websites I returned to it. I have a rather old account there (since 2018), but I haven’t touched it in a while. I have had 5 kyu since forever, now I really want to get 4 kyu, even though it is not just in Python hehe.

My modest achievements at Codewars

In February I upgraded my DS/Algorithms skills with Udemy courses, Yandex Youtube lectures and personal studies here and there. I am excited to return to Problem Solving katas at Codewars with all this acquired knowledge.

Summing Up

All in all, right now I am very much into coding simulators. They bring me a sense of accomplishment that I desperately need right now. Badges and certificates – small achievements you can touch. That brings comfort if anything. So I encourage everybody I know who struggles with reality right now – to do the same. At very least, you came out of it more educated and with brighter prospects.

GitPython – Diff between two branches

My dear brother started his own blog on Airflow, Python, PostgreSQL and Javascript!!!

Congrats, Ilia!

Ilia's Airflow, Python, PostgreSQL, Javascript Journey

Code:

import os
from pathlib import Path 
from git import Repo

# path to repo folder
path = Path(os.path.dirname(os.path.abspath(__file__)))

# repo variable
repo = Repo(path)

# Your last commit of the current branch
commit_feature = repo.head.commit.tree

# Your last commit of the dev branch
commit_origin_dev = repo.commit("origin/dev")
new_files = []
deleted_files = []
modified_files = []

# Comparing 
diff_index = commit_origin_dev.diff(commit_feature)

# Collection all new files
for file in diff_index.iter_change_type('A'):
    new_files.append(file)

# Collection all deleted files
for file in diff_index.iter_change_type('D'):
    deleted_files.append(file)

# Collection all modified files
for file in diff_index.iter_change_type('M'):
    modified_files.append(file)

Analyze:

Let’s imagine the situation:

You are before code review and you need to get all changes between feature and develop with GitPython.
First you need to get last commits from two branches:

# Your last commit of the current branch
commit_dev = repo.head.commit.tree

# Your last commit of the dev branch
commit_origin_dev = repo.commit("origin/dev")

Then get your changes:

View original post 81 more words

Google Drive API with Python – Upload & Download a file

For how to authenticate yourself please go watch this video.

You will need a token.json with this content in the root of your project:

{
  "token": "generated_access_token",
  "refresh_token": "generated_refresh_token",
  "token_uri": "https://oauth2.googleapis.com/token",
  "client_id": "your_client_id",
  "client_secret": "your_client_secret",
  "scopes": [
    "https://www.googleapis.com/auth/drive"
  ]
}

I will be using Google’s Quickstart as a foundation for what I am doing.

Read More »

How to add GIF to a static background (PIL + imageio) V.2 – The Better One

My previous version has flaws. It works well for opaque GIFs, but OpenCV is famous for not being good with transparent images. So instead of OpenCV I use imageio for GIF iteration and PIL for adding GIF frames to the static background.

Why do more?

The original algorithm left these artifacts, look at the cyan block on the intersection of GIF frame and a darker abstract part:

The artefact in question
Read More »

Video Generation Journey. Part 1: Making frames (problems 1-4)

Initially, I aimed to make one big post, but I am splitting it up. This Part 1 is about how I made the frames to generate videos from.

Intro

I have been on and off doing this project for a few months. Many things did not work for me – from ffmpeg to sheer OpenCV/ffmpeg installation on Apple M1 systems – and I am not 100% sure what made it work to this day. I plan to put myself in a situation where I have to do it again, and I will make a guide then. However, right now, I can only say that this video tutorial on running native ffmpeg and this guide from OpenCV themselves are good places to start.

The last time I used OpenCV was in 2017 when I worked on my graduate work. Back then, I processed prerecorded videos with no need to worry about audio. So I had some background prior to this project, but I was in no way ready for the ride.

Buckle up!

Table of contents

  • What do I have to create?
  • Limitations
    • Background
    • Phrases
    • Audio
  • Problems
    • Problem 1: GIF on a background
    • Problem 2: Putting text on the image
    • Problem 3: Text wrapping
    • Problem 4: Text blocks positioning
  • Results so far
Read More »

[Note] ‘ChatConsumer’ has no attribute ‘as_asgi’ – Sloppy development

Following Channels’ docs I encountered yet another error the solution to what was not obvious right off the bat.

Ditch Channels.

The End.

This was my original post in October 2020 and I stand by it.

You can make your project work with Google Firebase - it handles signals well enough. Yes it is primarily used by mobile developers, but overall it is fine for web too.

I spent trying to install Django Channels on my local Linux 18.04, and I kind of made it work, but then I had to deploy it... And as a professional backender you don't want to let people down.

I decided to try Channels because my Python-related friends talked about it. But days into the trouble I asked my friend about a certain step and he asked me in return "oh so [you use it because] they fixed Channels since two years ago?". I as like ??????????????????. So that moment I ditched it. Too much time wasted, but I like the result I achieved with Firebase.

What I am saying - Firebase is fast and universal. In a matter of days I managed to make and deploy an internal Customer Support chat service that connected mobile app users and a support from the web.</p>

I will be writing a new series on Django + Firebase, and this post is pretty much a short introduction.