v1.55 : remove warning message 10/04/2025
Voici les arguments acceptés par le crawler :
- del=1 : supprime le "cache" du crawler (vide la table).Utile pour relancer un crawl sans attendre 60 minutes.
- drop=1 : supprime complétement la table. Un peu plus efficace que del=1.
- supprimer_site_statique=1 : supprime les fichiers statiques du site, pour un gros reset. Plus efficace que del ou drop. ATTENTION VOIR AVEC SYLVAIN AVANT D'UTILISER CETTE OPTION. En effet, on peut perdre les pages profondes du site.
- nb=10 : va stopper le crawler après 10 crawls. Utile pour lire un message d'erreur ou autre sans attendre 3 minutes de chargement. Si on met pas nb=, ça crawl 2000 URL.
- debug=1 : affiche de l'aide au débuguage
- dont_copy_pictures=1 : Lance le crawler sans copier les images du dossier "upload"
- copy_all_pictures=1 : En cas de bug sur les images, utiliser cet argument. Le crawl va être plus long, mais toutes les images du dossier "upload" seront copiées.
- copy_js_and_css=1 : En cas de bug sur le css ou le JS, utiliser cet argument. Le crawl va être plus long, mais tous les CSS et JS du dossier "upload" seront copiées.
- info=1 : affiche des infos sur le crawler, sans exécuter le crawl
- lancer le crawler de façon normale, sans option
- gerer_htpasswd=non Le crawler ne créera pas de fichier htpasswd
- gerer_htaccess_www=non Le crawler ne créera pas de fichier htaccess pour rediriger le CP vers www, ou non-www.
- crawler_market.php?gerer_htaccess_www=oui&nb=1 >> Le crawler va forcer la création du htaccess, même s'il faut écraser l'ancien htaccess. Attention, re-créer le htaccess supprimera les éventuelles anciennes redirections 301 !
Remarque : Si pas d'argument, le crawler créera le htaccess uniquement s'il n'y en a pas. - Aller sur le site statique
- crawler_market.php?url=xxx&nb=1 => lancer seulement une url
- crawler_market.php?url=xxx&supprimer_page_statique_si_404=1&nb=1 => permet de supprimer la page statique si elle a été supprimée de wordpress
- crawler_market.php?url=xxx&nb=1&redirect_to_url=1 => le crawler redirigera vers l'URL statique quand il aura terminé
Gdoc d'aide au débugage, à disposition de toute l'équipe
tabledusite = wp_tfckmarorg
bddhostdusite = unkurpquser.mysql.db
bddnamedusite = unkurpquser
bdduserdusite = unkurpquser
bddpassworddusite = xxx
tabledusite = wp_tfckmarorg
Erreur, pas de ligne en BDD trouvée url vaut https://www.tfckmar.org/wp23a82bbc699d955/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/guerre-en-ukraine-la-russie-peut-elle-vraiment-tenir-dans-la-duree/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/category/non-classe/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/feed/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/comments/feed/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/css/dist/block-library/style.min.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/admin-block.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper-free/style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/bootstrap.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/main.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/default-style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper-free/assets/css/main.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/all.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/block.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/css/responsive.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/xmlrpc.php
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/jquery.min.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/jquery-migrate.min.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/imagesloaded.min.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/masonry.min.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/js/mobile-menu.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/js/ticker.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/js/scripts.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/author/admin235/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/guerre-en-ukraine-la-russie-peut-elle-vraiment-tenir-dans-la-duree/feed/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/8
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/oembed/1.0/embed
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/comment-reply.min.js
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/category/non-classe/feed/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/categories/1
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-brands-400.svg
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-regular-400.svg
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-solid-900.svg
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-brands-400.eot
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-brands-400.woff2
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-brands-400.woff
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-brands-400.ttf
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-regular-400.eot
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-regular-400.woff2
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-regular-400.woff
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-regular-400.ttf
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-solid-900.eot
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-solid-900.woff2
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-solid-900.woff
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/webfonts/fa-solid-900.ttf
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/r.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/u.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/this.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/e.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/n.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/le.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/ce.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/u.style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/l.style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/e.style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/t.style.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/:ce.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/:c.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/s.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/jQuery.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/:s.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/s.fn.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/jquery/jQuery.fn.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/this.img.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/this.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/d.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/js/self.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-content/themes/newsx-paper/assets/js/this.track.css
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/author/admin235/feed/
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/users/1
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-2.png\"
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-3.jpg\"
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-5.png\"
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-1.png\"
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-4.jpg\"
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-2.png
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-3.jpg
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-5.png
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-1.png
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-json/wp/v2/posts/\"https:\/\/www.tfckmar.org\/wp23a82bbc699d955\/wp-content\/uploads\/2025\/11\/guerre-4.jpg
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/:E.documentElement.cur
-------------------------------------------------
on va crawler URL : https://www.tfckmar.org/wp23a82bbc699d955/wp-includes/js/i.cur
On a déjà crawlé cette URL il y a moins d'une heure et on trouve pas d'autres URL, c'est peut être finit, exit now.
[crawl_success_ok]